Database integration
Database integration
Sometimes the only way to get data out of a system is to connect to its database directly: an older application without an API, an in-house system, or a reporting need that an API cannot serve efficiently. Direct database integration can be fast and powerful, but it bypasses the application's rules, ties you to its internal structure and can slow the live system. Done carefully, through read-only access, replicas and views, it is a legitimate tool; done carelessly, it causes subtle data problems.
This page explains when connecting directly to a database makes sense and how to do it safely.
Illustrative diagram
Database integration
- Access
- Read-only account
- Least privilege
- Separation
- Read replica
- Reporting store
- Stable interface
- Views
- Documented fields
- Changes
- Change data capture
- Scheduled extracts
API or direct database connection
Use the application's API
Suits you when
- An API exists that covers your needs
- You need to change data
Watch out for
- Rate limits and coverage gaps
Read directly from the database
Suits you when
- No API, or reporting needs large volumes
- Read-only access is enough
Watch out for
- Internal structure can change; load on the live system
Write directly to the database
Suits you when
- Rarely, with the application owner's agreement and no alternative
Watch out for
- Bypasses validation and business rules; high risk of inconsistent data
Safer ways to read from a database
| Technique | What it does |
|---|---|
| Read replica | A copy of the database kept in step, used for reading without loading the live one |
| Database views | A stable, documented shape of data that hides internal tables |
| Scheduled extracts | Data copied at set times into a reporting store |
| Change data capture | Changes read from the database's log and passed on as they happen |
| Read-only account | Access that cannot alter data |
The risks of writing directly
Applications enforce rules when they save data: required fields, calculated values, related records, audit trails and notifications. Writing directly into the database skips all of these, so records can end up inconsistent in ways the application does not expect, sometimes causing errors much later. Updates to the application can also change table structures without notice, breaking direct writes. If writing is unavoidable, agree it with whoever maintains the application, keep it minimal, test thoroughly and monitor the results.
Where possible, add a small API to the application instead of writing to its tables.
Use views as a contract
Application databases are designed for the application, not for outsiders. Table names, columns and relationships can change with any update. Creating database views that present the data you need in a stable, documented shape gives integrations something dependable to read. When the application changes, only the view needs updating, and every integration using it keeps working. Views also limit exposure, showing only the fields an integration needs rather than whole tables that may include sensitive data.
Agree the views with whoever maintains the application, so they know not to break them.
Protecting the live system
Heavy queries against a live database can slow the application for its users, especially reporting queries that scan large tables. Running them against a read replica, or on a schedule outside busy hours into a separate reporting store, keeps the live system responsive. Monitor query times and set limits so one integration cannot exhaust the database's resources.
Securing database access
Accounts
- Dedicated account per integration
- Read-only unless writing is agreed
- Access limited to required views or tables
Network
- Database not exposed publicly
- Encrypted connections
- Access from known locations only
Oversight
- Connections logged
- Credentials rotated
- Access reviewed when people or providers change
Change data capture
Many databases record every change in a log for their own recovery purposes. Change data capture tools read that log and turn each insert, update and delete into an event that other systems can consume, close to real time and without querying the tables repeatedly. It suits keeping a reporting store or search index up to date from a busy system. It needs careful setup and monitoring, and support depends on the database product and how it is hosted.
For simpler needs, a scheduled extract of records changed since the last run is often enough.
Legacy systems without APIs
Older systems are a common reason for database integration. Before connecting, understand the data well: which tables hold what, how records relate, which fields are actually used and what the codes mean. Long-serving staff and any surviving documentation help. Where the system will be replaced eventually, a database integration can bridge the gap; the legacy modernisation pages cover longer-term options.
Questions about database integration
When should we connect directly to a database?
When no suitable API exists or reporting needs large volumes, preferably read-only.
Is writing directly to a database safe?
Rarely; it bypasses validation and business rules and can create inconsistent data.
What is a read replica?
A copy of the database kept up to date for reading without loading the live system.
What is change data capture?
Reading changes from the database's log and passing them on as they happen.
Why use database views?
They give a stable shape of data that hides internal tables, reducing breakage.
What if the application is updated?
Internal structures may change; views and testing reduce the risk.
Request a Custom Quote
Ask about Database integration
Thank you — your enquiry has been sent.
We have received your message and will reply by email. If you need to add anything, email info@letgetgrow.com.