So, you want to search for information? Maybe it’s a Twitter message and LinkedIn profile related to your target. Or maybe you’re looking for internal documents leaked from your target’s organization. Either way, you’ll need some tools to use. In this article, I will demonstrate how to use open-source intelligence (osint) tools to find what organizations and individuals may not want you to know.
OSINT-Tools
👀 Some of my favorite OSINT tools.
If you want me to add something, just make a pull request.
If you want a list of resources rather than tools, you should check out this repository.
Tools
- Maltego – Open source graphical link analysis tool for gathering and connecting OSINT.
- Metagoofil – Extract metadata from popular file types.
- Recon-ng – For lots of web based recon.
- theHarvester – (in my opinion) A Better version of recon-ng.
- cree.py – A geolocation OSINT tool for social media.
- SpiderFoot – Open source footprinting and intelligence-gathering tool.
- XRay – Tool for mapping and OSINT gathering from public networks.
- trape – A cool people tracker written in python.
- Goohak – Automatically launch Google hacking queries against a target domain.
- The Infected Drake – A web-penetration testing toolkit, presently suited for reconnaissance purposes.
- Email2PhoneNumber – A script for attempting to find a phone number associated with an e-mail.
- ReconDog – A reconnaissance multi tool with a wide array of features.
- OWASP Amass – Get information from: DNS, web scraping, APIs, and web archives.
- iKy – Collects information on an e-mail address and displays it in a fancy UI.
- Moriarty – Collects information from phone numbers
- GHunt – GHunt is an OSINT tool to extract a lot of informations of someone’s Google Account email.
- Spyse – OSINT gathering platform that collects valuable data and stores it in its own database to provide info without scanning. Info clusters: IPv4 hosts, domains/whois/site info, ports/banners/protocols, technologies, maintain biggest SSL/TLS db, AS, OS etc…
Username Checkers
- Check Usernames
- No Name Username Scanner
- Gaddr
- KnowEm (Searches a lot of things)
The Top 757 Osint Open Source Projects on Github
Categories > Data Processing > OsintSherlock ⭐ 28,486🔎 Hunt down social media accounts by username across social networksTwint ⭐ 12,034An advanced Twitter scraping & OSINT tool written in Python that doesn’t use Twitter’s API, allowing you to scrape a user’s followers, following, Tweets and more while evading most API limitations.Ghunt ⭐ 10,479🕵️♂️ Investigate Google emails and documents.Social Analyzer ⭐ 8,433API, CLI & Web App for analyzing & finding a person’s profile across +1000 social media \ websites (Detections are updated regularly by automated systems)Pentesting Bible ⭐ 8,066Learn ethical hacking.Learn about reconnaissance,windows/linux hacking,attacking web technologies,and pen testing wireless networks.Resources for learning malware analysis and reverse engineering.Awesome Osint ⭐ 7,837😱 A curated list of amazingly awesome OSINTPhoton ⭐ 7,822Incredibly fast crawler designed for OSINT.Spiderfoot ⭐ 6,885SpiderFoot automates OSINT for threat intelligence and mapping your attack surface.Trape ⭐ 6,442People tracker on the Internet: OSINT analysis and research tool by Jose PinoAmass ⭐ 6,288In-depth Attack Surface Mapping and Asset DiscoveryTheharvester ⭐ 6,173E-mails, subdomains and names Harvester – OSINTPhoneinfoga ⭐ 5,927Information gathering & OSINT framework for phone numbersGitrob ⭐ 5,145Reconnaissance tool for GitHub organizationsSubfinder ⭐ 4,503Subfinder is a subdomain discovery tool that discovers valid subdomains for websites. Designed as a passive framework to be useful for bug bounties and safe for penetration testing.Singlefile ⭐ 4,406Web Extension for Firefox/Chrome/MS Edge and CLI tool to save a faithful copy of an entire web page in a single HTML fileAquatone ⭐ 4,369A Tool for Domain FlyoversOneforall ⭐ 4,127OneForAll是一款功能强大的子域收集工具Instaloader ⭐ 3,588Download pictures (or videos) along with their captions and other metadata from Instagram.Digital Privacy ⭐ 3,586Information Protection & OSINT resources | 一个关于数字隐私搜集、保护、清理集一体的方案,外加开源信息收集(OSINT)对抗Rengine ⭐ 3,435reNgine is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines, recon data correlation and organization, continuous monitoring, backed by a database, and simple yet intuitive User Interface. reNgine makes it easy for penetration testers to gather reconnaissance with minimal configuration and with the help of reNgine’s correlation, it just makes recon effortless.Osmedeus ⭐ 3,388Fully automated offensive security framework for reconnaissance and vulnerability scanningOsint Framework ⭐ 3,331OSINT FrameworkDnstwist ⭐ 3,107Domain name permutation engine for detecting homograph phishing attacks, typo squatting, and brand impersonationShhgit ⭐ 2,995Ah shhgit! Find secrets in your code. Secrets detection for your GitHub, GitLab and Bitbucket repositories: www.shhgit.comOsintgram ⭐ 2,981Osintgram is a OSINT tool on Instagram. It offers an interactive shell to perform analysis on Instagram account of any users by its nicknameDiscover ⭐ 2,546Custom bash scripts used to automate various penetration testing tasks including recon, scanning, parsing, and creating malicious payloads and listeners with Metasploit.Ivre ⭐ 2,333Network recon framework, published by @cea-sec & @ANSSI-FR. Build your own, self-hosted and fully-controlled alternatives to Shodan / ZoomEye / Censys and GreyNoise, run your Passive DNS service, collect and analyse network intelligence from your sensors, and much more!Raccoon ⭐ 2,305A high performance offensive security tool for reconnaissance and vulnerability scanningHttpx ⭐ 2,299httpx is a fast and multi-purpose HTTP toolkit allows to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.Opencti ⭐ 2,166Open Cyber Threat Intelligence PlatformIntelowl ⭐ 2,115Intel Owl: analyze files, domains, IPs in multiple ways from a single API at scaleH8mail ⭐ 2,005Email OSINT & Password breach hunting tool, locally or using premium services. Supports chasing down related emailXray ⭐ 1,647XRay is a tool for recon, mapping and OSINT gathering from public networks.Aleph ⭐ 1,538Search and browse documents and data; find the people and companies you look for.Tidos Framework ⭐ 1,386The Offensive Manual Web Application Penetration Testing Framework.Pagodo ⭐ 1,365pagodo (Passive Google Dork) – Automate Google Hacking Database scraping and searchingTinfoleak ⭐ 1,311The most complete open-source tool for Twitter intelligence analysisHacking Resources ⭐ 1,290Hacking resources and cheat sheets. References, tools, scripts, tutorials, and other resources that help offensive and defensive security professionals.Snoop ⭐ 1,213Snoop — инструмент разведки на основе открытых данных (OSINT world)Phishing_catcher ⭐ 1,205Phishing catcher using CertstreamGitgraber ⭐ 1,205gitGraber: monitor GitHub to search and find sensitive data in real time for different online services such as: Google, Amazon, Paypal, Github, Mailgun, Facebook, Twitter, Heroku, Stripe…Urlhunter ⭐ 1,157a recon tool that allows searching on URLs that are exposed via shortener servicesHolehe ⭐ 1,146holehe allows you to check if the mail is used on different sites like twitter, instagram and will retrieve information on sites with the forgotten password function.Pwnedornot ⭐ 1,134OSINT Tool for Finding Passwords of Compromised Email AddressesAwesome Asset Discovery ⭐ 1,108List of Awesome Asset Discovery ResourcesIntrigue Core ⭐ 1,097Discover Your Attack Surface!Torbot ⭐ 1,074Dark Web OSINT ToolReconspider ⭐ 1,032🔎 Most Advanced Open Source Intelligence (OSINT) Framework for scanning IP Address, Emails, Websites, Organizations.Sn0int ⭐ 1,030Semi-automatic OSINT framework and package managerOsint_collection ⭐ 1,030Maintained collection of OSINT related resources. (All Free & Actionable)Gitgot ⭐ 1,029Semi-automated, feedback-driven tool to rapidly search through troves of public data on GitHub for sensitive secrets.Infoga ⭐ 1,021Infoga – Email OSINTAwesome Social Engineering ⭐ 1,004A curated list of awesome social engineering resources.Blackwidow ⭐ 986A Python based web application scanner to gather OSINT and fuzz for OWASP vulnerabilities on a target website.Paramspider ⭐ 986Mining parameters from dark corners of Web ArchivesHarpoon ⭐ 877CLI tool for open source and threat intelligenceMaigret ⭐ 789🕵️♂️ Collect a dossier on a person by username from thousands of sitesAttacksurfacemapper ⭐ 733AttackSurfaceMapper is a tool that aims to automate the reconnaissance process.Git Hound ⭐ 669Reconnaissance tool for GitHub code search. Finds exposed API keys using pattern matching, commit history searching, and a unique result scoring system.Favfreak ⭐ 664Making Favicon.ico based Recon Great again !Mitaka ⭐ 662A browser extension for OSINT searchWhatsmyname ⭐ 650This repository has the unified data required to perform user enumeration on various websites. Content is in a JSON file and can easily be used in other projects.Powerful Plugins ⭐ 646Powerful plugins and add-ons for hackersSocialscan ⭐ 638Python library and CLI for accurately querying username and email usage on online platformsGasmask ⭐ 617Information gathering tool – OSINTCloud_enum ⭐ 609Multi-cloud OSINT tool. Enumerate public resources in AWS, Azure, and Google Cloud.Email2phonenumber ⭐ 599A OSINT tool to obtain a target’s phone number just by having his email addressOsrframework ⭐ 598OSRFramework, the Open Sources Research Framework is a AGPLv3+ project by i3visio focused on providing API and tools to perform more accurate online researches.Hosthunter ⭐ 560HostHunter a recon tool for discovering hostnames using OSINT techniques.Osi.ig ⭐ 558Information Gathering Instagram.Metabigor ⭐ 552Intelligence tool but without API keyLinkedin2username ⭐ 547OSINT Tool: Generate username lists for companies on LinkedInInstagramosint ⭐ 546An Instagram Open Source Intelligence ToolMaryam ⭐ 516Maryam: Open-source Intelligence(OSINT) FrameworkVajra ⭐ 511Vajra is a highly customizable target and scope based automated web hacking framework to automate boring recon tasks and same scans for multiple target during web applications penetration testing.Xeuledoc ⭐ 505Fetch information about a public Google document.Whatbreach ⭐ 496OSINT tool to find breached emails, databases, pastes, and relevant informationWatcher ⭐ 493Watcher – Open Source Cybersecurity Threat Hunting Platform. Developed with Django & React JS.Mihari ⭐ 492A framework for continuous OSINT based threat huntingGoohak ⭐ 487Automatically Launch Google Hacking Queries Against A Target DomainOpen Semantic Search ⭐ 484Open Source research tool to search, browse, analyze and explore large document collections by Semantic Search Engine and Open Source Text Mining & Text Analytics platform (Integrates ETL for document processing, OCR for images & PDF, named entity recognition for persons, organizations & locations, metadata management by thesaurus & ontologies, search user interface & search apps for fulltext search, faceted search & knowledge graph)Odin ⭐ 482Automated network asset, email, and social media profile discovery and cataloguing.Awesome Termux Hacking ⭐ 478⚡️An awesome list of the best Termux hacking toolsBigbountyrecon ⭐ 471BigBountyRecon tool utilises 58 different techniques using various Google dorks and open source tools to expedite the process of initial reconnaissance on the target organisation.Bugcrowd Levelup Subdomain Enumeration ⭐ 464This repository contains all the material from the talk “Esoteric sub-domain enumeration techniques” given at Bugcrowd LevelUp 2017 virtual conferenceThreatingestor ⭐ 454Extract and aggregate threat intelligence.Operative Framework ⭐ 451operative framework is a OSINT investigation framework, you can interact with multiple targets, execute multiple modules, create links with target, export rapport to PDF file, add note to target or results, interact with RESTFul API, write your own modules.Witnessme ⭐ 445Web Inventory tool, takes screenshots of webpages using Pyppeteer (headless Chrome/Chromium) and provides some extra bells & whistles to make life easier.Fav Up ⭐ 444IP lookup by favicon using ShodanNamechk ⭐ 443Osint tool based on namechk.com for checking usernames on more than 100 websites, forums and social networks.Sonarsearch ⭐ 430A rapid API for the Project Sonar datasetMoriarty Project ⭐ 430This tool gives information about the phone number that you entered.Awesome Telegram Osint ⭐ 417📚 A Curated List of Awesome Telegram OSINT Tools, Sites & ResourcesGosint ⭐ 411OSINT Swiss Army KnifeOsint Brazuca ⭐ 407Repositório criado com intuito de reunir informações, fontes(websites/portais) e tricks de OSINT dentro do contexto Brasil.Toutatis ⭐ 403Toutatis is a tool that allows you to extract information from instagrams accounts such as e-mails, phone numbers and moreEnterprise Security Skill ⭐ 403用于记录企业安全规划,建设,运营,攻防的相关资源Asn ⭐ 400ASN / RPKI validity / BGP stats / IPv4v6 / Prefix / URL / ASPath / Organization / IP reputation and geolocation lookup tool / Traceroute serverBelati ⭐ 399The Traditional Swiss Army Knife for OSINTSitedorks ⭐ 390Search Google/Bing/Ecosia/DuckDuckGo/Yandex/Yahoo for a search term (dork) with a default set of websites, bug bounty programs or custom collection.
Conclusion
In this article, I want to share with you my experience in using social media data for intelligence. For several years, I was researching hand-curated datasets from multiple sources and selling them on demand. It was technically difficult to get it all done, so I decided to automate the process.