• Monitoring the Entire Web with Google Custom Search

    Written By: Idan Aharoni      Nov 16, 2018
    Whether the goal of your web intelligence activities is to detect cybercrime, identify threats to your organization or collect insights on your competitors, one of the most powerful and useful tools available is Google Search. While the average person uses Google Search to find r... Read More
       , ,
  • Pastebin Monitoring Methods

    Written By: Idan Aharoni      Oct 5, 2018
    When we start working with a new customer, we usually receive a wish-list of sources that they wish to monitor using our platform. Based on the sources indicated on the list, we set up the coverage of the system. One of the sources that keeps coming up on these wish-lists, reques... Read More
       , ,
  • Enabling TOR Access for Crawlers and Scripts Using Privoxy

    Written By: Idan Aharoni      Apr 29, 2018
    In recent years, monitoring .onion sites, which are sites hosted on The Onion Router (TOR) network, has become a necessity for most web intelligence operations. So many bad sites are hosted on TOR, such as cybercriminal forums and drug marketplaces, that it is often mistaken for ... Read More
       , ,
  • HTTP Proxy Setup with Squid on Ubuntu 16.04

    Written By: Idan Aharoni      Apr 10, 2018
    Proxies are an imperative tool in web intelligence work. They enable the collection of data from sensitive locations on the web such as the Dark Web, and use avatars in social media and other closed resources. Proxies can be obtained for free from available lists, bought from var... Read More