On BaseCNPJ, you start typing a company name and suggestions appear instantly — over a base of more than 62 million establishments. It looks trivial from the user side; from the engineering side, it is a classic problem of indexing and normalizing public data. Here is how Meta Dados solves it.
The challenge: autocomplete over 62 million records
As-you-type search must answer in tens of milliseconds, or the suggestion arrives after the next keystroke. Doing that with a wildcard LIKE on a 62-million-row table is the guaranteed path to slowness: every keystroke scans the whole base.
Indexing for prefix search
The way out is to prepare the data for search, not to search the raw data. We keep an index layer optimized for prefix and term matching, over the already-normalized legal and trade names. The query the user fires hits the index — small and built for it — not the entire transactional table.
Normalization: Brazilian open data is messy
Brazilian Federal Revenue open data arrives with inconsistent accents, mixed case, abbreviations and noise. Before indexing, we normalize: accent stripping, case standardization, handling of common terms. That normalization is what lets "padaria sao joao" find "PADARIA SÃO JOÃO LTDA" without the user worrying about how they typed it.
Geolocation by exact address
Here is the differentiator few have: we do not stop at the municipality. A self-hosted geocoder on a dedicated server resolves the full registration address into coordinates (latitude/longitude), cached per entity. That is already more than 27.5 million geolocated points, covering 99.7% of the geocodable population — the remainder being companies based abroad, characterized and bounded, not ignored. This enables real proximity and radius queries, sector density and territorial coverage — not a city-center approximation.
Freshness: reindex on every Revenue cycle
The Revenue publishes data in monthly cycles. On each cycle, the base is reprocessed and the index rebuilt — with completeness validation before going live. And Metabase tracks the usage of the search and API per customer, closing the loop between engineering and business.
Frequently asked questions
How do you suggest companies as the user types without overloading the database?
The key is not to search the transactional table on every keystroke. An index layer prepared for prefix matching, over already-normalized fields, answers in milliseconds and isolates search from the production load. The main database stays free for what it does best.
Is the geolocation the exact coordinate of each company?
Yes. A self-hosted geocoder running on a dedicated server resolves the full registration address into coordinates (latitude/longitude), cached per entity — over 27.5 million points, 99.7% coverage of the geocodable population. What we do not geocode (companies based abroad) is characterized and bounded, not hidden.