• Learn & Support
    • Learn and support
        • Resource Hub
          Access value added content to support your IP strategy
        • Webinars & Events
          Are you interested in attending one of our online or onsite event?
        • Product Trainings
          Customer success is our priority. Increase your skills in the use of Questel’s software
        • Product News
          A platform dedicated to software and platforms news and evolutions
        • eLearning Programs
          Create engaging and custom corporate training programs
  • About Questel
    • About questel
        • Mission
          Helping our clients to succeed in the modern knowledge-based economy
        • Leadership
          A passionate group of leaders united in their commitment to advance technology and talent
        • History
          More than 18 acquisitions during the past 5 years, strategic partnerships and product launches
        • Technology
          Big Data, Artificial Intelligence and Blockchain are in our DNA
        • Offices
          28 offices around the globe, local expertise and global reach
        • Corporate Social Responsibility
          A critical investment for long-term success
        • Certifications
          Continuous improvement for better quality, security and positive impact
        • Social Projects
          We intend to create a better world for all of us
        • Team Building
          Together with everyone's effort
  • Careers →
    • Careers
    • Login Pages →
      EN
      questel_logo
      • Patent
        • Our Patent solutions
            • → IP Intelligence Software
            • Orbit Intelligence
              Powerful patent searching and analysis
            • AI-Classifier
              Artificial intelligence for patent classification
            • Prosecution Pack
              Grant statistics and detailed legal status analysis
            • Chemistry
              Small molecules searching and analysis
            • Orbit Biosequence
              DNA and amino acid searching and analysis
            • Orbit Insight
              Explore new markets and opportunities
            • Orbit Express
              Easy patent searching and collaboration
            • Orbit Intellixir
              Analysis from any scientific sources
            • → AI meets IP, your final true companion
              Artificial Intelligence will become increasingly pervasive in our everyday life.
            • → IP Management Software
            • Equinox Corporate
              Powerful all-in-one IPMS for corporates
            • Equinox Corporate+
              IPMS tailored for large corporates, built on Salesforce
            • Equinox Law Firm
              Powerful all-in-one IPMS for law firms
            • Equinox Law Firm+
              IPMS tailored for large Law Firms
            • Equinox Invention
              Collaborative invention-to-filing workflow
            • Equinox Blockchain
              Blockchain-based timestamping and protection
            • Equinox eBilling
              Simplified IP invoice management
            • → Patent Services
            • Search
              Patentability, Freedom-To-Operate, Invalidity
            • International Filing Support
              PCT & national phase, Paris Convention, Unitary Patent
            • Unitary Patent Support
              Unitary Patent and Unified Patent Court services
            • EP Validation
              Simplified European validation services
            • Translation
              Centralized patent translation services
            • Renewals
              Fee and renewals management
            • Recordals
              Assignment and recordal of all IP Rights
            • → Beyond Traditional Prior Art Searching
              Watch this on-demand webinar to learn everything you need to know about SEP, BioSequ...
            • → Patent Strategy & Administration
            • Administrative Support
              End-to-end IP admin, data verification and docketing
            • Invoice Management
              Invoice review, bundling and payment
            • Cost Management
              Fee audits, negotiation, compliance, benchmarks and forecasting
            • IP Consulting
              Landscaping, benchmarking, valuation and licensing
            • Defensive Publication
              Publish an invention into the public domain
            • → Get up to speed on the latest in IP Invoice and Cost Management
              You are instructing directly or via your local IP law firms foreign associates acros...
      • Trademark
        • Our Trademark solutions
            • → Clearance & Watch Platform
            • Markify ProSearch™
              Screen trademarks with AI-driven search technology
            • Markify Comprehensive Search
              AI-driven trademark availability software
            • Markify Watch
              Watch for new, identical and confusingly similar trademarks
            • Markify Brand Monitor
              Monitor marketplaces, social media, and web content
            • → Developing a Robust Trademark Search and Watching Program
              The trademark lifecycle contains multiple points at which something can go wrong wit...
            • → IP Management Software
            • Equinox Corporate
              Powerful all-in-one IPMS for corporates
            • Equinox Corporate +
              IPMS tailored for large corporates, built on Salesforce
            • Equinox Law Firm
              Powerful all-in-one IPMS for law firms
            • Equinox Law Firm+
              IPMS tailored for large law firms
            • Equinox Blockchain
              Blockchain-based timestamping and protection
            • Equinox Brand Proposal
              Collaborative worklfow for trademark and design proposals
            • → Mastering Brand Projects: Simplifying the Process for Optimal Results and Success
              Watch this on-demand webinar to learn how you can simplify and enrich your workflow....
            • → Trademark, Design & Domain Services
            • Search
              Analyst-driven searches with on-demand level of reporting
            • Watch
              IP Assets monitoring with intelligent pre-select service
            • Online Brand Protection
              Case management and take-down services
            • Filing
              International registration process support
            • Renewals
              Trademark and Design renewal services
            • Domain Name Management
              Effective management of corporate domain names
            • Recordals
              Assignment and recordal of all IP Rights
            • → Alternative Legal Service Providers vs Law Firms: What’s the best choice?
              An Alternative Legal Service Provider provides specialized services in high-demand l...
            • → Trademark Strategy & Administration
            • Administrative Support
              End-to-end IP admin, data verification, docketing
            • Invoice Management
              Invoice review, bundling and payment
            • → Streamline Your Trademark Renewal Process: Leverage Technology to Reduce Internal Burdens
              Trends in managing Trademark renewals are numerous. Today, more and more companies a...
      • Innovation
        • Our Innovation solutions
            • → Innovation Management Software
            • Innosabi Insight
              Business-oriented dashboards for innovators
            • Innosabi Startup
              Startup deal flow and program management
            • Innosabi Idea
              Collaborative idea and innovation management
            • Innosabi Community
              Customers co-creation and communities
            • Innosabi Partner
              Partners co-innovation and open calls
            • Innosabi Tech
              Collaborative tech knowledge management
            • → What is Innovation Scouting?
              Change is a constant. New technologies surface regularly and change existing product...
            • → Innovation Services
            • Innovation Consulting
              Market screening, Competitive intelligence and Technology scouting
            • Operational Services
              Connect with partners, improve or localize your innovation process
            • Naming
              Products and services naming
            • Software Localization
              Seamless user experience to a global audience
            • Marketing Translation
              Engage with new audience globally
            • → Patenting Software: How can it be done?
              Even though life without computers is today nearly impossible to imagine, patent-bas...
      • Legal Operations
        • Our Legal operations solutions
            • → Legal Operations Software
            • Ascent ELM
              Comprehensive Enterprise Legal Management
            • Ascent eDiscovery
              Best-in-class eDiscovery solutions, services, and support
            • Ascent Blockchain
              Blockchain-based timestamping and protection
            • → Top 5 Benefits of ELM Solutions Boost Legal Ops Value
              Learn how the top 5 benefits delivered by ELM solutions help Legal Ops meet evolving...
            • → Legal Operations Services
            • ELM Data Integration
              Seamless data source integration
            • Legal Support Services
              Support services to enhance your in-house capabilities
            • Multilingual Litigation Support 
              Fast and reliable certified legal translation services
            • Legal Interpretation
              Precise and reliable legal interpretation services
            • Legal Translation
              Reliable and convenient legal translation services
            • → The Secrets to ELM Success – Featuring Emerson’s Legal Ops Testimonial
              Business needs evolve, and so do your Legal department responsibilities. In the era ...
      • Localization
        • Our Localization Solutions
            • → Life Science
            • Regulatory Translation Services 
              Quality translations that mitigate risk
            • Clinical Translation and Validation
              High-quality translation and linguistic validation solutions
            • Medical Interpretation
              Reliably accurate interpretation for patients and providers
            • → Corporate
            • Technical Translation
              Remain compliant and ensure safe use of your products with quality technical translation
            • eLearning Localization
              Increase efficacy and learning comprehension with quality eLearning localization services
            • Marketing Translation
              Extend your reach and engage new audiences with high-quality marketing translation services
            • Software Localization
              Create seamless user experiences across the globe with experienced localization engineers
            • Website Localization
              Reach audiences in new markets with accurate website localization services
            • Multimedia Localization
              Create global multi-channel content with convenient multimedia localization services
            • → Legal
            • Multilingual Litigation Support
              Fast and reliable certified legal translation services
            • Legal Translation
              Reliable and convenient legal translation services
            • Legal Interpretation
              Precise and reliable legal interpretation services
            • Patent Translation
              Centralized patent translation services
      • Contact
      questel_logo
      EN
      questel-menu
      closeClose
      • Contact
        menu-bracket
      • Patent
        menu-bracket
        • IP Intelligence Software
          menu-arrow
          IP Intelligence Software
        • Orbit Intelligence
          Powerful patent searching and analysis
          menu-bracket
        • AI-Classifier
          Artificial intelligence for patent classification
          menu-bracket
        • Prosecution Pack
          Grant statistics and detailed legal status analysis
          menu-bracket
        • Chemistry
          Small molecules searching and analysis
          menu-bracket
        • Orbit Biosequence
          DNA and amino acid searching and analysis
          menu-bracket
        • Orbit Insight
          Explore new markets and opportunities
          menu-bracket
        • Orbit Express
          Easy patent searching and collaboration
          menu-bracket
        • Orbit Intellixir
          Analysis from any scientific sources
          menu-bracket
        • IP Management Software
          menu-arrow
          IP Management Software
        • Equinox Corporate
          Powerful all-in-one IPMS for corporates
          menu-bracket
        • Equinox Corporate+
          IPMS tailored for large corporates, built on Salesforce
          menu-bracket
        • Equinox Law Firm
          Powerful all-in-one IPMS for law firms
          menu-bracket
        • Equinox Law Firm+
          IPMS tailored for large Law Firms
          menu-bracket
        • Equinox Invention
          Collaborative invention-to-filing workflow
          menu-bracket
        • Equinox Blockchain
          Blockchain-based timestamping and protection
          menu-bracket
        • Equinox eBilling
          Simplified IP invoice management
          menu-bracket
        • Patent Services
          menu-arrow
          Patent Services
        • Search
          Patentability, Freedom-To-Operate, Invalidity
          menu-bracket
        • International Filing Support
          PCT & national phase, Paris Convention, Unitary Patent
          menu-bracket
        • Unitary Patent Support
          Unitary Patent and Unified Patent Court services
          menu-bracket
        • EP Validation
          Simplified European validation services
          menu-bracket
        • Translation
          Centralized patent translation services
          menu-bracket
        • Renewals
          Fee and renewals management
          menu-bracket
        • Recordals
          Assignment and recordal of all IP Rights
          menu-bracket
        • Patent Strategy & Administration
          menu-arrow
          Patent Strategy & Administration
        • Administrative Support
          End-to-end IP admin, data verification and docketing
          menu-bracket
        • Invoice Management
          Invoice review, bundling and payment
          menu-bracket
        • Cost Management
          Fee audits, negotiation, compliance, benchmarks and forecasting
          menu-bracket
        • IP Consulting
          Landscaping, benchmarking, valuation and licensing
          menu-bracket
        • Defensive Publication
          Publish an invention into the public domain
          menu-bracket
      • Trademark
        menu-bracket
        • Clearance & Watch Platform
          menu-arrow
          Clearance & Watch Platform
        • Markify ProSearch™
          Screen trademarks with AI-driven search technology
          menu-bracket
        • Markify Comprehensive Search
          AI-driven trademark availability software
          menu-bracket
        • Markify Watch
          Watch for new, identical and confusingly similar trademarks
          menu-bracket
        • Markify Brand Monitor
          Monitor marketplaces, social media, and web content
          menu-bracket
        • IP Management Software
          menu-arrow
          IP Management Software
        • Equinox Corporate
          Powerful all-in-one IPMS for corporates
          menu-bracket
        • Equinox Corporate +
          IPMS tailored for large corporates, built on Salesforce
          menu-bracket
        • Equinox Law Firm
          Powerful all-in-one IPMS for law firms
          menu-bracket
        • Equinox Law Firm+
          IPMS tailored for large law firms
          menu-bracket
        • Equinox Blockchain
          Blockchain-based timestamping and protection
          menu-bracket
        • Equinox Brand Proposal
          Collaborative worklfow for trademark and design proposals
          menu-bracket
        • Trademark, Design & Domain Services
          menu-arrow
          Trademark, Design & Domain Services
        • Search
          Analyst-driven searches with on-demand level of reporting
          menu-bracket
        • Watch
          IP Assets monitoring with intelligent pre-select service
          menu-bracket
        • Online Brand Protection
          Case management and take-down services
          menu-bracket
        • Filing
          International registration process support
          menu-bracket
        • Renewals
          Trademark and Design renewal services
          menu-bracket
        • Domain Name Management
          Effective management of corporate domain names
          menu-bracket
        • Recordals
          Assignment and recordal of all IP Rights
          menu-bracket
        • Trademark Strategy & Administration
          menu-arrow
          Trademark Strategy & Administration
        • Administrative Support
          End-to-end IP admin, data verification, docketing
          menu-bracket
        • Invoice Management
          Invoice review, bundling and payment
          menu-bracket
      • Innovation
        menu-bracket
        • Innovation Management Software
          menu-arrow
          Innovation Management Software
        • Innosabi Insight
          Business-oriented dashboards for innovators
          menu-bracket
        • Innosabi Startup
          Startup deal flow and program management
          menu-bracket
        • Innosabi Idea
          Collaborative idea and innovation management
          menu-bracket
        • Innosabi Community
          Customers co-creation and communities
          menu-bracket
        • Innosabi Partner
          Partners co-innovation and open calls
          menu-bracket
        • Innosabi Tech
          Collaborative tech knowledge management
          menu-bracket
        • Innovation Services
          menu-arrow
          Innovation Services
        • Innovation Consulting
          Market screening, Competitive intelligence and Technology scouting
          menu-bracket
        • Operational Services
          Connect with partners, improve or localize your innovation process
          menu-bracket
        • Naming
          Products and services naming
          menu-bracket
        • Software Localization
          Seamless user experience to a global audience
          menu-bracket
        • Marketing Translation
          Engage with new audience globally
          menu-bracket
      • Legal Operations
        menu-bracket
        • Legal Operations Software
          menu-arrow
          Legal Operations Software
        • Ascent ELM
          Comprehensive Enterprise Legal Management
          menu-bracket
        • Ascent eDiscovery
          Best-in-class eDiscovery solutions, services, and support
          menu-bracket
        • Ascent Blockchain
          Blockchain-based timestamping and protection
          menu-bracket
        • Legal Operations Services
          menu-arrow
          Legal Operations Services
        • ELM Data Integration
          Seamless data source integration
          menu-bracket
        • Legal Support Services
          Support services to enhance your in-house capabilities
          menu-bracket
        • Multilingual Litigation Support 
          Fast and reliable certified legal translation services
          menu-bracket
        • Legal Interpretation
          Precise and reliable legal interpretation services
          menu-bracket
        • Legal Translation
          Reliable and convenient legal translation services
          menu-bracket
      • Localization
        menu-bracket
        • Life Science
          menu-arrow
          Life Science
        • Regulatory Translation Services 
          Quality translations that mitigate risk
          menu-bracket
        • Clinical Translation and Validation
          High-quality translation and linguistic validation solutions
          menu-bracket
        • Medical Interpretation
          Reliably accurate interpretation for patients and providers
          menu-bracket
        • Corporate
          menu-arrow
          Corporate
        • Technical Translation
          Remain compliant and ensure safe use of your products with quality technical translation
          menu-bracket
        • eLearning Localization
          Increase efficacy and learning comprehension with quality eLearning localization services
          menu-bracket
        • Marketing Translation
          Extend your reach and engage new audiences with high-quality marketing translation services
          menu-bracket
        • Software Localization
          Create seamless user experiences across the globe with experienced localization engineers
          menu-bracket
        • Website Localization
          Reach audiences in new markets with accurate website localization services
          menu-bracket
        • Multimedia Localization
          Create global multi-channel content with convenient multimedia localization services
          menu-bracket
        • Legal
          menu-arrow
          Legal
        • Multilingual Litigation Support
          Fast and reliable certified legal translation services
          menu-bracket
        • Legal Translation
          Reliable and convenient legal translation services
          menu-bracket
        • Legal Interpretation
          Precise and reliable legal interpretation services
          menu-bracket
        • Patent Translation
          Centralized patent translation services
          menu-bracket
      More
    • Learn & Support
      menu-bracket
      • Resource Hub
        menu-bracket
      • Webinars & Events
        menu-bracket
      • Product Trainings
        menu-bracket
      • Product News
        menu-bracket
      • eLearning Programs
        menu-bracket
    • About Questel
      menu-bracket
      • Mission
        menu-bracket
      • Leadership
        menu-bracket
      • History
        menu-bracket
      • Technology
        menu-bracket
      • Offices
        menu-bracket
      • Corporate Social Responsibility
        menu-bracket
      • Certifications
        menu-bracket
      • Social Projects
        menu-bracket
      • Team Building
        menu-bracket
    • Careers
      menu-bracket
    • Login Pages
      menu-bracket
    • How Domain Name Monitoring Works
      Blog Post / Published, June 17, 2021

      How Domain Name Monitoring Works

      Back to Resource hub
      Back to Resource hub
      overview

      Over the years, domain monitoring has become  a standard industry practice, and an important component of most online brand protection services. Nonetheless, there are a few questions about the inner workings of domain name monitoring that we keep on getting asked:

      • How do Domain Name Crawlers work?
      • Why are mentions of a brandname in a subdomain not detected?
      • Why are some domain monitoring software not crawling certain ccTLDs?

      These are some of the questions we address in this article.

      Let’s start with a definition:

      Domain Monitoring is the act of searching for relevant domains in domain registry databases.

      The easy part: What you consider a relevant domain depends on your objective. As we are discussing domain monitoring in a brand protection context, relevant here means a risk to your brand e.g. those domains that infringe your trademark rights. If you are using domain monitoring as part of your competitive intelligence, relevance would doubtlessly have a different meaning.

      Let us now look at the other part of the definition. What are domain registry databases , and how we can search in them?

      In order to keep track of who (when and for how long?)  has registered a domain, registries keep databases that store this information. Whether and how we can search those registry databases depends on whether the registries give access to the public. Here, not all domain endings are equal. Let us elaborate a bit.

      Domain names can be divided into two groups: Those that are governed by ICANN (generic Top Level Domains, gTLDs) and those that are governed by national governments (country code Top Level Domains, ccTLDs). It’s easy to separate them. All two character TLDs are ultimately under the governance of countries, known examples are .cn, .ru, .de, uk., while all TLDs with 3 or more characters such as .com, .info, .shop, .app, .swiss follow regulations set out by ICANN.

      This setup creates a domain world where there are unified rules regarding registry databases for all gTLDs and a fragmented landscape for ccTLDs.

      Luckily, many ccTLDs voluntarily follow ICANN processes for many aspects of TLD governance but not nearly all.

      This is important to start with because it explains why getting access to registry databases is not as straightforward as it may seem.

       

      How do Domain Name Crawlers work?

      Let’s start with the case of databases that are accessible by the public.

      Whenever a domain is licensed to a registrant, the registrar marks the domain as registered in the registry database, together with other pertinent information, such as the registration date, expiry and registrant data. From that point on the public can look up the WHOIS for the domain. Domain monitoring tools search through a list of all domains registered since the bots last crawled, which we call here newly registered domains. The list of domains you find in your domain monitoring tool results page is a subset of all newly registered domains. Exactly which domains end up on your results page depends on the way your crawler is set to filter the results, but most brands are interested in finding domains that include their trademark, or domains that look confusingly similar to their trademark.

      Filtering newly registered domains happens via an algorithm using so called “boolean operators”(AND,OR,NOT). You are familiar with those from any monitoring tool. Lines such as does not include keywords (negative keyword) help you minimize the amount of “false positive” results, data which is actually irrelevant even though it contains your trademark.

      Example: The car brand Jaguar may want to exclude all results that include “zoo”, “safari” or other keywords that are very likely to be associated with something other than the car brand.

      Besides the Boolean Operators, domain crawlers are also using algorithms that identify domain names which may be confusingly similar to your brand, without containing an exact match of your brand name. A domain name can be confusingly similar to your brand on different levels including phonetically and visually.

      Phonetic similarity

      The french words “deux mains” (two hands) and demain (tomorrow) sound very similar, even though the words appear quite different visually. When you are reading a word which is phonetically similar to a brand, you may automatically make a connection to that brand, especially if followed by a brand-related keyword. In practice, this is rarely used in scams, because it’s rather difficult to come up with good examples, so most people are underwear of it. 😛

      Visual proximity

      A much more commonly used way of creating proximity is by means of “look”. When reading we mainly focus on the first and last letters of a word, followed by the existence of characters we would expect to find in a word while the exact order of the characters is less relevant for our understanding of a word (see Typoglycemia).

      Cybersquatters take advantage of this cognitive trait by registering domains that appear to be a known brand, while there is actually a small difference such as an omitted letter or additional letter, a double letter, a typo on a visually similar letter from the same or even from a different script.

      Jaquar.com - Can you spot the mistake?

      Depending on the brand you are monitoring for, the algorithm has either a strict or loose configuration. You want to catch all relevant results and omit as many false positives as possible to save analyst time when filtering through the result list.

      How does domain name monitoring for ccTLDs differ?

      ccTLDs on the other hand are not obliged to make the registered domains available to the public. This means that not all ccTLD databases can be crawled actively and other heuristic techniques are applied to monitor domains that contain keywords such as your brand under ccTLDs.

      One such heuristic is to check whether a set of potentially interesting domains is registered under the TLD. For example, the monitoring tool could use domains that have been registered in gTLDs and contain your brand as a sample, then check whether any of those names are unavailable under the ccTLD in question. If the domain is unavailable, this is an indication that this particular domain is registered and therefore may need further attention from your brand protection team.

      The heuristic ccTLD monitoring approach is less robust than searching through public databases because registries may block monitoring tools if they receive too many requests, others require a license. It is therefore laborious to finetune towards the different registries leading to a trade-off between cost of adding an extra ccTLD and the value you will get from monitoring less common extensions such as Ethiopian or Burmese domain registrations.

      Therefore, domain monitoring software is always including gTLDs but how many ccTLDs are included differs a lot from provider to provider. It is up to each brand to assess whether the extra domain monitoring coverage is worth the extra expense.

      Why are Subdomains not included in conventional Domain Monitoring?


       

      It is important to understand where exactly domain monitoring crawlers are searching to understand the limitations of their results.

      Let’s have a look at the structure of a URL:

      protocol://sub-domain.second-level-domain.TLD/Folder1/Folder2

      Protocol: Specifies the process of how computers communicate with each other. In most instances this will be http or https.

      Sub-Domain: As admin of a domain, one can create as many and whatever kind of sub-domains one wants. Some commonly used subdomains are www, mail, test, app. Subdomains can be used to point to different servers and therefore services. You might be accessing “drive.google.com” or “docs.google.com” running under different subdomains of google.com.

      Second-level-Domain:  The part that you can register to get an exclusive license to use the name.

      Top-Level Domain or First-Level Domain (TLD): The TLD follows the second-level domain and delimits a space under the governance of the TLD registry.

      /FolderStructure: Directs a query to the right folder and file to fetch the requested data under that URL, also called Path.

      Which of the URL components is covered by domain monitoring crawlers?

      Crawlers will only look at the domain name part, because that’s the only part that is available in the searchable registry database.

      In other words, sub-domains and the path or folder structure are not analyzed by conventional domain monitoring solutions.

       

      At Questel we offer a customized domain monitoring service balancing effectiveness and cost. If you are looking to protect your brand from abuse within domain names, let’s have a chat and explore your needs.

      Questel

      A world leader in intellectual property, innovation and legal management.

      • Patent
        • IP Intelligence Software
        • IP Management Software
        • Patent Services
        • Patent Strategy & Administration
      • Trademark
        • Clearance & Watch Platform
        • IP Management Software
        • Trademark, Design & Domain Services
        • Trademark Strategy & Administration
      • Innovation
        • Innovation Management Software
        • Innovation Services
      • Legal Operations
        • Legal Operations Software
        • Legal Operations Services
      • Localization
        • Life Science
        • Corporate
        • Legal
      Stay in touch!
      Subscribe to our newsletters to receive all of the latest news, updates and amazing offers.
      © 2023 Questel. All rights reserved
      • Legal notice