BaseCNPJ is Meta Dados' proprietary enrichment and commercial intelligence platform: registration, corporate and tax data on Brazilian companies, queryable through a real-time API. The database consolidates over 62 million business establishments, built from Brazilian Federal Revenue (RFB) open data — reprocessed in a heavy monthly ingestion cycle — and combined with our own enrichment layers.
A database this size only creates value if you can see it. We needed a BI layer to answer business questions — coverage by state and economic activity, data quality and freshness, API usage per customer — without building dashboards from scratch for every new question.
The challenge
- Heavy analytical queries (aggregations by economic activity, state, company size and registration status) without degrading the production API.
- Public data is notoriously messy: every cycle brings sudden layout changes in the RFB files, broken encodings and inconsistent records — such as invalid or non-existent economic activity codes (CNAE) that break referential integrity.
- Sales and product teams with no SQL skills needing self-service analytics.
- Monitoring ingestion: every RFB load requires completeness and consistency validation.
The solution
We deployed self-hosted Metabase (Docker, kept on the latest version) as the BI layer of BaseCNPJ:
- Architecture — a separate database acting strictly as a data warehouse on DigitalOcean, isolating the heavy analytical workload from the production API that serves customers.
- Modeling — PostgreSQL materialized views with the most-queried cuts (economic activity × state × size × status), combined with native Metabase Models into a semantic layer business users understand.
- Quality — post-load validation pipelines check completeness and consistency on every new RFB cycle.
- Dashboards — database coverage and freshness, API usage funnel per customer (over 1.6 million queries tracked, ~23k per day), and data quality metrics.
- Self-service — the business and product teams build complex reports visually with the query builder, with no dependency on engineering.
Results
Questions that previously required advanced SQL — or days of development — are now answered through self-service in seconds. The sales team independently filters active companies by economic activity, region, share capital and registration date, building market segments for high-conversion outbound campaigns without opening a ticket with engineering.
The stack
PostgreSQL · Metabase (self-hosted, Docker) · Python ingestion pipelines · DigitalOcean · Brazilian Federal Revenue open data.
Why Metabase
Open source and self-hosted — full control over infrastructure and data, essential under LGPD (Brazil's data privacy law); a query builder business teams actually use; and pricing that scales with team size, not data volume.
Frequently asked questions
Why not use the API itself for reporting?
The API is built for fast transactional lookups, not heavy aggregations. Running analyses by economic activity, state and size directly on the production database would degrade response times for API consumers. That is why we split the analytical layer into a dedicated data warehouse.
Do business users need to know SQL?
No. Metabase's query builder lets them assemble segments and dashboards without writing a single line of SQL — and power users still have the SQL editor available.