justia-crawler
A web crawler for justia, the patent searching website
Science Score: 31.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (8.1%) to scientific vocabulary
Repository
A web crawler for justia, the patent searching website
Basic Info
- Host: GitHub
- Owner: SuperKenVery
- Language: Python
- Default Branch: main
- Size: 18.6 KB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
Justia Crawler
Gets patent data from justia.
Features
- Crawl all patents of a company from justia
- Cache all fetched HTMLs so you can easily debug without being banned
- Lazily load each single page
- Lazily load each property with cache
Setup
I'm too lazy to upload this thing to pypi, so you'll have to clone the sources and develop in this folder. Therefore, this is essentially a "development" setup.
We use pixi to manage the python environment, which creates a reproducible environment with locked package versions.
To install dependencies,
```bash
Ensure you have pixi installed
pixi install ```
then you could run the example with:
bash
pixi run python patent_crawler.py
Usage
The bottom of patent_crawler.py contains a simple example. patents_after_2020.py is also a reference too.
python3
import patent_crawler
patents = patent_crawler.get_all_patents("meta-platforms-inc")
for p in patents:
print(p.title)
print(p.abstract)
# See more properties in source code
Owner
- Login: SuperKenVery
- Kind: user
- Repositories: 6
- Profile: https://github.com/SuperKenVery
Citation (citation_analyze.py)
#!/usr/bin/env python3
import patent_crawler
from typing import Dict, List, Tuple
from rich import traceback
traceback.install(show_locals=True)
from rich.progress import track
from tqdm import tqdm
import gc
def analyze_citations(company: str) -> Dict[str, int]:
print("Filtering out patents after 2020")
patents = list(filter(lambda x: x.file_date.year>=2020, patent_crawler.get_all_patents(company)))
length = len(patents)
del patents
cited_times: Dict[str, int] = {}
patents = filter(lambda x: x.file_date.year>=2020, patent_crawler.get_all_patents(company))
for patent in track(patents, total=length, description=company):
for cite in patent.citations:
if cite.patent_id not in cited_times:
cited_times[cite.patent_id] = 1
else:
cited_times[cite.patent_id] += 1
return cited_times
def first_cite_rate(company: str) -> float:
cited_times = analyze_citations(company)
if len(cited_times)==0: return 0
cited_once = len(list(filter(lambda x: x==1, cited_times.values())))
return cited_once / len(cited_times)
def main():
meta = "meta-platforms-inc"
openai = "openai-opco-llc"
amazon = "amazon-technologies-inc"
anthropic = "anthropics-technology-limited"
for company in meta, openai, amazon, anthropic:
# for company in amazon, anthropic:
print(f"First cite rate for {company}: {first_cite_rate(company)}")
if __name__ == "__main__":
main()
GitHub Events
Total
- Push event: 6
- Create event: 2
Last Year
- Push event: 6
- Create event: 2