The Challenge
A prominent global real estate firm faced a significant challenge: Data fragmentation from over 8,000 diverse property assets spread across multiple locations. Existing systems led to inefficiencies and inaccuracies in data retrieval and reporting, impeding strategic decision-making. With each asset generating arrays of data – including occupancy rates, maintenance schedules, lease details, and financial metrics – the absence of a unified data framework hampered comprehensive analysis and proactive management.
The Solution
In response to this challenge, Adyantrix developed a cutting-edge PropTech data platform designed to unify fragmented data sources into a streamlined, accessible system. Our team implemented an integrated data pipeline that aggregated information from numerous property management systems across the portfolio into a centralised cloud-based data repository. This platform utilised API integration and robust ETL (Extract, Transform, Load) processes to ensure seamless data flow.
To enhance usability, we incorporated sophisticated data analytics tools and interactive dashboards offering real-time insights into property performance metrics. Our solution leveraged advanced machine learning algorithms for predictive analytics, helping the real estate firm foresee property market trends and optimize asset management.
Key features of the platform included automated data cleansing and enrichment, which significantly reduced manual errors and ensured data accuracy. Furthermore, scalable cloud infrastructure supported high data volumes and future expansion needs.
Key Results
The PropTech data platform delivered a transformative impact:
- Reduced Data Redundancy: By consolidating data from 8,000 property assets, redundancies were reduced by 75%, enhancing data reliability.
- Improved Decision-Making: Real-time analytics provided a 60% increase in the speed of generating actionable insights, allowing the firm to act promptly on investment and management opportunities.
- Operational Efficiency Gains: The platform streamlined reporting processes, cutting the average report generation time by 50% and freeing up resource allocation toward strategic initiatives.
- Enhanced Data Accuracy: With automated cleansing processes, data accuracy improved by 90%, leading to more precise market analysis and asset evaluation.
Adyantrix's tailored PropTech solution established a single source of truth for the firm's asset data, driving operational efficiency and strategic foresight in their extensive real estate operations.
Technical Approach
The platform was built on a modern data lakehouse architecture, with Microsoft Azure as the primary cloud provider given the client's existing enterprise agreements. The core components were:
- Azure Data Factory orchestrated all ingestion pipelines, connecting to 23 distinct source systems — including Yardi Voyager, MRI Software, CoStar data feeds, IoT building sensors, and bespoke legacy databases built on MS SQL Server and Oracle — via a combination of REST APIs, JDBC connectors, and flat-file SFTP ingestion for older systems that lacked API capability.
- Azure Data Lake Storage Gen2 served as the raw and curated data lake, with a bronze-silver-gold medallion architecture separating raw ingestion, cleansed and conformed data, and business-ready aggregated tables.
- Azure Databricks (Apache Spark) handled all large-scale data transformation and the machine learning feature pipelines, processing the daily delta of approximately 4.2 million records across the portfolio without performance degradation.
- Azure Synapse Analytics provided the SQL-accessible semantic layer consumed by reporting tools, enabling analysts to query across the entire portfolio using familiar T-SQL syntax without needing to understand the underlying lake structure.
- Power BI Premium was used for all interactive dashboards, with row-level security ensuring that regional asset managers could only view data for their assigned geographies.
Data governance was implemented using Microsoft Purview, with a unified data catalogue that tagged every dataset with its source system, data owner, refresh cadence, and applicable data-retention policy — a critical requirement for the firm's compliance obligations under GDPR across its European portfolio.
Implementation Highlights
The engagement ran over 28 weeks, structured across three major phases with bi-weekly steering committee reviews.
Phase 1 — Source System Audit and Schema Design (Weeks 1–8): The team catalogued all 23 source systems, documenting data quality issues, schema inconsistencies, and referencing conflicts. A significant challenge was the absence of a consistent property identifier across systems: different databases used different codes for the same physical asset. We designed and implemented a Master Asset Registry (MAR) — a golden-record matching process using fuzzy address matching and postcode lookups — that assigned a canonical AssetID to each of the 8,000 properties, which then became the universal join key across the platform.
Phase 2 — Pipeline Build and Data Cleansing (Weeks 9–20): ETL pipelines were built in Azure Data Factory using a parameterised template approach, meaning that once the first five source connectors were built and tested, subsequent connectors could be configured from metadata rather than requiring bespoke development. Automated data quality rules — 740 in total — were implemented in the silver layer using Great Expectations, flagging anomalies such as lease expiry dates in the past, negative occupancy rates, and duplicate asset records for human review rather than silently passing corrupt data downstream.
Phase 3 — Analytics Layer and Rollout (Weeks 21–28): Fifteen Power BI dashboard templates were built covering portfolio occupancy, lease expiry concentration risk, maintenance cost trends, and capital expenditure forecasting. Dedicated onboarding sessions were run with 85 users across six regional offices, including a self-service training portal so that new joiners could onboard independently post-launch.
Measurable Outcomes
The business impact of the platform became measurable within the first 90 days of full production operation:
- The 75% reduction in data redundancy was quantified by comparing the number of conflicting records for the same asset across systems before and after the Master Asset Registry was applied — from an average of 3.1 conflicting records per asset to 0.8, representing a near-elimination of the duplication that had previously corrupted portfolio-level reporting.
- Report generation time fell from an average of 6.4 hours (involving manual data pulls from multiple systems and spreadsheet consolidation) to under 45 minutes on the new platform — a reduction that freed approximately 2,400 analyst-hours per year across the reporting team.
- The predictive maintenance model — trained on 36 months of historical maintenance cost and IoT sensor data — identified 127 assets with elevated mechanical failure risk in its first quarter of operation, enabling proactive intervention that the client's facilities team estimated avoided £340,000 in emergency maintenance costs.
- The firm's investment committee adopted the platform's portfolio risk dashboard as its primary briefing tool for quarterly asset review meetings, replacing a previously manual 40-page slide deck that had been prepared by a dedicated analyst over two weeks before each meeting.
Lessons Learned
Building a platform of this scale across a geographically dispersed portfolio surfaced several lessons that inform how we approach large-scale data integration engagements:
- The Master Asset Registry is the hardest and most important piece. Every technical capability of the platform — joins, aggregations, trend analysis — depends on reliably resolving which records in different systems refer to the same physical asset. Allocating dedicated resource to this problem in Phase 1, rather than treating it as a background task, was the single most impactful architectural decision of the project.
- Parameterised pipeline templates reduce cost at scale. The initial investment in building a reusable connector framework paid for itself from the eighth source system onwards. Had we built bespoke pipelines for each of the 23 sources, the development cost and ongoing maintenance burden would have been materially higher.
- Data governance needs a business owner, not just a technical one. The Microsoft Purview data catalogue was technically complete at launch, but adoption was initially low. Assigning a named data steward in each regional office — responsible for reviewing flagged quality issues and approving new data definitions — transformed the catalogue from a compliance artefact into a living tool that the business actively maintained.
Speak with our Data Engineering team at Adyantrix to find out how we can support your next project.
Work with Adyantrix
If you are looking to tackle a similar challenge, Adyantrix has the expertise to help across the full project lifecycle. Our data engineering practice covers pipeline design, streaming, and data infrastructure. Our data analytics practice covers BI reporting and self-serve analytics platforms. Our data visualisation practice covers interactive dashboards and visual reporting. Our business intelligence practice covers BI strategy, platform selection, and dashboard delivery. Get in touch to discuss your requirements — no commitment required.
