Recent work

Write-ups

Clients aren't named, and nor is anything that would identify them. The problems, and how they were found, are real.

Migration

Two planning systems into one, with an eightfold costing error caught first

8×how far too high project costs would have read if the rates had loaded as they were

Client: Professional services firm
Move: Two project and timesheet systems into one cloud platform
Scope: Nine object types, years of history
Approach: SQL warehouse first, load second

Users, projects, memberships, tasks, assignments, time entries and time off all had to move, and the rules behind them had drifted apart between the two source systems over the years.

Everything was extracted into a SQL warehouse first, so each rule could be tested with a query rather than checked by eye. Migration templates were filled from validated views, never from raw tables.

Before any load, a validation suite checks for orphaned records, invalid emails, blank required fields, duplicate external IDs, impossible dates, broken references and tasks with no parent. Time off was trimmed to what actually needed to move, to stay inside the pilot's record limit.

The client's leadership received a data quality report: each issue, its root cause, and how to keep the new platform clean.

What the data showed

Time entries above the target platform's 24-hour daily cap.
Archived records the target treats as read-only, which a straight load would have failed on.
Gaps in the skills taxonomy between the two systems.
Cost rates held as blended daily figures in one system and hourly in the other. Loaded as they were, project costs would have read roughly eight times too high.
A margin report in the existing BI that spread annual salary across booked days instead of hours worked.

Reverse engineering

Reading 32,000 lines of undocumented VBA, straight from the file

113modules pulled out of the workbook file with a purpose-built parser, and all of them read

Client: Housing association
System: In-house Excel and VBA application
Scope: 113 modules, around 32,000 lines
Way in: A custom parser for the workbook file

Before deciding whether to keep, fix or replace the application, the association needed to know what was inside it. We wrote a parser for the workbook's file format, pulled out every VBA module, and read all of it.

The result was an executive review the leadership team could act on, and a fixed-price plan to replace the application with a supported ASP.NET Core and React web application.

How we turn spreadsheets like this into systems

What the code showed

SQL built by joining strings together, open to injection.
Money held in Single, a floating-point type that cannot hold every penny exactly.
Application state tracked by two flags that could disagree with each other.
Cached formula errors carried forward into calculations.
A worksheet password stored in plain text.
Automation built on Internet Explorer, which Microsoft has retired.

Warehousing, before Bit Movers

Running a global firm's Azure data warehouse, tuned and cheaper to run

2 yearsleading the data team at a global cyber security consultancy, before Bit Movers was set up

Employer: Global cyber security consultancy
Role: Senior data team lead
Platform: Azure data warehouse and Power BI
When: 2023 to 2025, in-house

This one is from before Bit Movers. Dan led the data team at a global cyber security consultancy and was the single point of contact for its reporting, analytics and data infrastructure.

The warehouse was business critical. Its queries were tuned, its tables partitioned and its resources managed, which improved performance and brought the cloud bill down.

Power BI reporting was built on top for the wider organisation. Earlier, as senior data analyst at the same firm, Dan designed and built integrations for expense management and sensitive HR data, and fixed and migrated data directly in the databases when the business needed it.

What the work involved

Query tuning to improve warehouse performance.
Partitioning strategies for the large tables.
Resource management to reduce cloud infrastructure costs.
Power BI reporting for decision-making across the business.
Integrations for expense management and sensitive HR data.
Best practice for data management and infrastructure design, and mentoring for junior engineers.

Elsewhere

Migrations out of ERP, CRM, PSA and document management platforms, for clients in cybersecurity, social housing, manufacturing and professional services.

Describe your systems

Tell us what you are moving from and to, and how the data can be reached. No data or credentials at this stage: a description is enough.

Describe your systems[email protected]