Abstract
This paper details the implementation of a Web crawler aimed at automating ruleset construction for “HTTPS Everywhere,” with a goal to convert HTTP URLs to secure HTTPS equivalents for enhanced communication security. Developed within a seven-month timeframe, the crawler faced challenges in verifying HTTPS support, varying based on SSL certificate existence and validity. Successful ruleset creation and testing in Firefox and Chrome, adhering to stylistic standards, demonstrated the potential for effective development. The paper explores improving productivity through alternative libraries like Scrapy and Scrapy Cloud. While certain goals, such as in-depth cryptocurrency analysis and web crawler background reading, were unmet due to time constraints, valuable insights were gained. The conclusion underscores the difficulties, successes, and promises of automating ruleset generation through web crawlers for “HTTPS Everywhere,” offering valuable recommendations for advancing web security.
| Original language | English |
|---|---|
| Pages (from-to) | 1-14 |
| Number of pages | 14 |
| Journal | International Journal of Information Security and Privacy |
| Volume | 18 |
| Issue number | 1 |
| DOIs | |
| Publication status | Published - 2024 |
| Externally published | Yes |
Bibliographical note
Copyright the Author(s) 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.Keywords
- HTTPS Everywhere
- Web Crawler
- Web Security
- SSL Certificate
- Rulesets
- Firefox
- Chrome
Fingerprint
Dive into the research topics of 'Automated ruleset generation for “HTTPS Everywhere”: challenges, implementation, and insights'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver