Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

push down url validation to avoid security check false alarm #2685

Merged
merged 14 commits into from
Oct 9, 2023
Merged

Conversation

lxning
Copy link
Collaborator

@lxning lxning commented Oct 5, 2023

Description

Please read our CONTRIBUTING.md prior to creating your first pull request.

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

The url validation is already done before we use the url. There is false security alarm b/c security check lost the context. To avoid the false alarm, we push down url validation.

Fixing:

  1. disable loading TS_ALLOWED_URLS from env
  2. move url validation close to "new URL(url)" to clearly show security false positive"
  3. sanity check archiveName to guard Uncontrolled data used in path expression

Fixes #(issue)
#2665

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A
    Logs for Test A

  • Test B
    Logs for Test B

Checklist:

  • Did you have fun?
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

@lxning lxning requested a review from msaroufim October 5, 2023 22:23
@codecov
Copy link

codecov bot commented Oct 5, 2023

Codecov Report

Merging #2685 (941171a) into master (f57240f) will not change coverage.
Report is 1 commits behind head on master.
The diff coverage is n/a.

❗ Current head 941171a differs from pull request most recent head 378dd50. Consider uploading reports for the commit 378dd50 to get more accurate results

@@           Coverage Diff           @@
##           master    #2685   +/-   ##
=======================================
  Coverage   72.39%   72.39%           
=======================================
  Files          85       85           
  Lines        3956     3956           
  Branches       58       58           
=======================================
  Hits         2864     2864           
  Misses       1088     1088           
  Partials        4        4           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@lxning lxning self-assigned this Oct 6, 2023
@lxning lxning added the security label Oct 6, 2023
msaroufim
msaroufim previously approved these changes Oct 6, 2023
@msaroufim msaroufim self-requested a review October 6, 2023 03:15
@msaroufim msaroufim dismissed their stale review October 6, 2023 03:15

let's run the code scanner after rebasing or merging to master

+ "AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY or AWS_DEFAULT_REGION");
}

HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();

Check failure

Code scanning / CodeQL

Server-side request forgery Critical

Potential server-side request forgery due to a
user-provided value
.
"Miss environment variables "
+ "AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY or AWS_DEFAULT_REGION");
URL endpointUrl = new URL(url);
FileUtils.copyURLToFile(endpointUrl, modelLocation);

Check failure

Code scanning / CodeQL

Server-side request forgery Critical

Potential server-side request forgery due to a
user-provided value
.
"Miss environment variables "
+ "AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY or AWS_DEFAULT_REGION");
URL endpointUrl = new URL(url);
FileUtils.copyURLToFile(endpointUrl, modelLocation);

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
.
throw new IOException(
"Security alert slash or backslash appear in archiveName:" + archiveName);
}

Copy link
Member

@msaroufim msaroufim Oct 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's some more validaiton we can do

  • Ensure the url is using the https protocol and not http or ftp
  • Potentially check that the IP is not a litteral IP and instead a domain?
  • Limit redirections
  • ..%2F is used for path traveral attacks
  • Optionally some additional checks on user-agent and content-type

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these validation is covered by function validateURL

@msaroufim msaroufim self-requested a review October 9, 2023 19:33
@lxning lxning added this pull request to the merge queue Oct 9, 2023
Merged via the queue into master with commit b41d9da Oct 9, 2023
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
v0.9.0 lifecycle
Awaiting triage
Development

Successfully merging this pull request may close these issues.

None yet

2 participants