-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Collect xen #464
Merged
Merged
Collect xen #464
Changes from all commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
c585f02
Merge pull request #2 from nexB/main
Pushpit07 a2e4487
Merge branch 'nexB:main' into main
Pushpit07 46aacf1
Added code to import Xen
Pushpit07 c5118c9
Merge branch 'main' into collect_xen
pombredanne 3f776ff
Merge branch 'main' into collect_xen
Pushpit07 307ead3
Update importer_yielder.py
Pushpit07 7b828ea
Added xen license info
Pushpit07 73a884f
helper: split_markdown_front_matter
Hritik14 a1752bd
black -l 100
Hritik14 614e0f3
Use split_markdown_front_matter helper in istio
Hritik14 d872763
Use a more obvious version and reorder imports
Hritik14 f4ca861
Update according to first review
Hritik14 617700a
Comment regarding raw docstring and sort imports
Hritik14 0f6eece
Remove added_advisories for updated_advisories
Hritik14 b891459
Speed up test_upstream
Hritik14 1f2cb59
:see_no_evil: Ignore legacy pypi package versions in github importer
sbs2001 1f8ef6b
Add basic implementation for time travel in imports
sbs2001 e847303
Use web scraping to obtain github tags
sbs2001 ba9051d
Time travel for githubtags, maven version api importers
sbs2001 9842367
Fix tests
sbs2001 0dccf22
Use class for returning versions partitioned by cutoff time
sbs2001 f7dc3e6
Make review changes
sbs2001 72a84bc
Refactor package_manangers
Hritik14 bece29b
Move load_api to parent class and refactor imports
Hritik14 80e3cf0
Remove misleading frozen dataclass
sbs2001 9d359f2
Fix npm importer date manipulation
sbs2001 047398e
Fix recursion in GitHubTagsAPI
Hritik14 9053ba9
Update importers to handle VersionResponse
Hritik14 1f3fdef
Update importer_yielder.py
Pushpit07 5774649
Added xen license info
Pushpit07 62c2b74
Merge branch 'collect_xen' of https://github.com/Pushpit07/vulnerable…
Pushpit07 bd18abe
Merge branch 'main' into collect_xen
Hritik14 8298b4c
Merge branch 'main' of github.com:nexb/vulnerablecode into collect_xen
Hritik14 6ef62fa
Rename DataSource to Improver, blackify, ignore tests
Hritik14 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
# Copyright (c) nexB Inc. and others. All rights reserved. | ||
# http://nexb.com and https://github.com/nexB/vulnerablecode/ | ||
# The VulnerableCode software is licensed under the Apache License version 2.0. | ||
# Data generated with VulnerableCode require an acknowledgment. | ||
# | ||
# You may not use this software except in compliance with the License. | ||
# You may obtain a copy of the License at: http://apache.org/licenses/LICENSE-2.0 | ||
# Unless required by applicable law or agreed to in writing, software distributed | ||
# under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR | ||
# CONDITIONS OF ANY KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations under the License. | ||
# | ||
# When you publish or redistribute any data created with VulnerableCode or any VulnerableCode | ||
# derivative work, you must accompany this data with the following acknowledgment: | ||
# | ||
# Generated with VulnerableCode and provided on an 'AS IS' BASIS, WITHOUT WARRANTIES | ||
# OR CONDITIONS OF ANY KIND, either express or implied. No content created from | ||
# VulnerableCode should be considered or used as legal advice. Consult an Attorney | ||
# for any legal advice. | ||
# VulnerableCode is a free software code scanning tool from nexB Inc. and others. | ||
# Visit https://github.com/nexB/vulnerablecode/ for support and download. | ||
|
||
import bz2 | ||
import dataclasses | ||
import json | ||
|
||
import requests | ||
from packageurl import PackageURL | ||
|
||
from vulnerabilities.importer import Importer | ||
from vulnerabilities.importer import Advisory | ||
from vulnerabilities.importer import Reference | ||
from vulnerabilities.helpers import create_etag | ||
from vulnerabilities.helpers import is_cve | ||
|
||
|
||
class XenImporter(Importer): | ||
CONFIG_CLASS = XenDBConfiguration | ||
|
||
def updated_advisories(self): | ||
advisories = [] | ||
if create_etag(data_src=self, url=self.config.db_url, etag_key="etag"): | ||
advisories.extend(self.to_advisories(fetch(self.config.db_url))) | ||
|
||
return self.batch_advisories(advisories) | ||
|
||
def create_etag(self, url): | ||
etag = requests.head(url).headers.get("etag") | ||
if not etag: | ||
return True | ||
|
||
elif url in self.config.etags: | ||
if self.config.etags[url] == etag: | ||
return False | ||
|
||
self.config.etags[url] = etag | ||
return True | ||
|
||
@staticmethod | ||
def to_advisories(xen_db): | ||
advisories = [] | ||
for xsa in xen_db[0]["xsas"]: | ||
reference = get_xen_references(xsa["xsa"]) | ||
title = xsa.get("title", [""]) | ||
for cve in xsa.get("cve", [""]): | ||
if not is_cve(cve): | ||
cve = "" | ||
|
||
advisories.append( | ||
Advisory( | ||
vulnerability_id=cve, | ||
summary=title, | ||
references=[reference], | ||
) | ||
) | ||
return advisories | ||
|
||
|
||
def get_xen_references(xsa_id): | ||
return Reference( | ||
reference_id="XSA-" + xsa_id, | ||
url="https://xenbits.xen.org/xsa/advisory-{}.html".format(xsa_id), | ||
) | ||
|
||
|
||
def fetch(url): | ||
response = requests.get(url).content | ||
return json.loads(response) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you know what the license would be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GPLv2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Pushpit07
if this is it, then state it. Do you have a link to the license info?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://wiki.xenproject.org/wiki/Xen_FAQ_General