Automation

Integrations: CRM, email, warehouse

Double data entry is a sure sign that systems aren't connected. Where exchange usually breaks, what to do when it fails and why it isn't a one-off job.

Most companies already run several systems: a website with requests, a CRM, email, inventory or service records, accounting. Each works fine on its own. The problem arises at the joints — where data should pass from one to another but passes through a person with a clipboard. Let's look at where exactly the link breaks and what to do about it.

Double entry as a symptom

If an employee enters the same data twice, it's not a matter of discipline but a missing integration. The signs are easy to spot from outside:

  • a request from the website arrives by email, and the CRM record is created by hand;
  • stock is kept in the accounting system, and on the website it's updated by an export once a day or after a phone call;
  • an invoice is issued in one program, and the payment status is marked in another;
  • a client list for mailings is assembled by exporting to a spreadsheet and uploading to the email service.

The cost of double entry isn't just time spent. It's guaranteed to create discrepancies: a phone number with a typo in one system, an order cancelled in another, and the product still shown as in stock on the website. Then comes dealing with the consequences, which costs more than the entry itself.

A separate effect is losing the source. When a request is transferred by hand, the campaign tag and visitor identifier stay in the email and never reach the CRM. This breaks end-to-end analytics at the very first step.

Where the link usually breaks

The places where exchange breaks are fairly uniform.

Different ideas of the same entity. In the CRM, a client is a contact with a phone number. In the accounting system, it's a counterparty with company details. On the website, it's an account with an email. Linking the records requires a common key. If it wasn't defined in advance, matching is done by name or phone, and sooner or later duplicates appear.

Different formats. A phone number with and without a plus, a date in two variants, amounts with and without cents, units of measurement. During testing everything matches; on real data from different sources, it doesn't.

Required fields. The receiving system requires a field the sending system simply doesn't have. The exchange fails on every other record, and nobody notices until a client complains.

External system limits. Almost every service has a limit on the number of requests. A one-off export of a thousand records can hit it and stop halfway.

Changes on the other side. A service updated its exchange interface, changed its response format or revoked an access key. Your code didn't change — but it stopped working.

Synchronisation and its conflicts

The key question of any integration: which direction does data flow, and which system is the master for each field?

One-way exchange is simpler and more reliable. A request from the website goes to the CRM, nothing comes back — there are no conflicts. Two-way sync is an order of magnitude harder, because there's now a case where a record was changed in both systems between exchange sessions.

Such situations need to be resolved in advance, not sorted out after the fact. Options:

  1. Assign a source of truth for each field. Stock — only from the accounting system; contact details — only from the CRM; price — only from the price list. A change in the non-master system is either forbidden or overwritten.
  2. Let the last change win. Simple to implement, but leads to unnoticed data loss: someone's change silently disappears.
  3. Record the conflict and call a person. Slow, but nothing is lost. Appropriate where the cost of a mistake is high.

The second important question is the exchange mode. Event-based exchange (the system reports a change itself) gives almost immediate up-to-date data but requires the receiving side to be available. Scheduled exchange is simpler and more robust, but data always lags a little. For stock levels this delay can be critical; for reporting it isn't.

And a third that people forget: reprocessing. The network blinked, a message was sent twice. If the exchange can't recognise an already processed record, you'll get duplicate orders. Every transferred operation needs an identifier by which it can be recognised on a retry.

An integration without protection against duplicates and without a conflict resolution rule works exactly until the first network glitch.

What to do when exchange fails

A failure isn't an exceptional situation but a normal operating mode for a bundle of several systems. Design with that in mind.

What should be in place:

  • An operation log. What was sent, when, with what result. Without a log, investigating an incident becomes guesswork.
  • A queue and retries. A failed transfer isn't discarded but deferred and retried at increasing intervals.
  • A notification to the person responsible. People should learn about a problem from the system, not from a client.
  • Manual retry. The ability to rerun a specific operation once the cause has been fixed.
  • Reconciliation. A periodic check that record counts and key totals in the two systems match. Reconciliation is what catches silent discrepancies that never produced a single error.

A separate note on access. Keys and tokens expire, passwords change, and an integration account can lose its rights during an access review. You should know in advance where these keys are stored and who renews them — access and security questions intersect directly with automation here.

Why integration isn't a one-off job

A bundle of systems lives in a changing environment. External services change, your own process changes, new fields are added, a new warehouse or a second legal entity appears. An integration handed over and forgotten degrades on its own.

In practice this means an integration needs:

  • an assigned owner on the business side and on the contractor side;
  • monitoring that reports a failed exchange without users having to;
  • documentation: which fields are transferred, in which direction, by what rule;
  • a test environment where changes are checked before going live;
  • resources budgeted for support, not just for development.

The last point is most often ignored at the agreement stage and then discussed in emergency mode. It's more honest to budget support from the start — it's part of the cost of ownership, just like hosting.

Checklist before starting

  • All systems are listed, and it's defined which data flows between them.
  • A source of truth is assigned for each field.
  • A common key is chosen for matching records between systems.
  • The exchange mode is defined: event-based or scheduled, and how often.
  • Behaviour on conflict and on repeated transfer is described.
  • There's a log, a retry queue and a notification to the person responsible on failure.
  • Regular data reconciliation is set up.
  • It's known where access keys are stored and who renews them.
  • The budget has a line for ongoing support.

If your systems are already connected but regularly drift apart, start with reconciliation and a log: they show the real picture faster than any audit. You can discuss your task here, and see how we approach this kind of work in the automation section.