Skip to content

Automation for generating the file on a nightly basis? #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
travisbotello opened this issue Jan 22, 2023 · 1 comment
Open

Automation for generating the file on a nightly basis? #12

travisbotello opened this issue Jan 22, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@travisbotello
Copy link

Hey there! Many thanks for this awesome script. Is it somehow possible auto generate those files every night and add it to this github repo? For example like: https://github.com/sapics/ip-location-db

@brianleect brianleect added the enhancement New feature or request label Jan 25, 2023
@brianleect
Copy link
Owner

brianleect commented Jan 25, 2023

Hmm I'm not too sure tbh since there's heavy reliance on logging in via etherscan first using selenium.

Main challenge would be

  • Clearing Captcha + automation for etherscan login

Alternative I suppose could be to leave an instance logged in running and simply scrape everything from the last pages of each recorded column.

Not sure about the compute costs though and perhaps a soln that relies on attaching cookies #9 would be much more efficient or a completely separate library used to controlling browser scraping might be better?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants