Working with telecommunications network data using FME Platform
About the project
Learn how Globema’s solution based on FME, a data processing and integration platform, supported a telecommunications company, Nexera. Our solution helped Nexera automate their data validation process, fix errors, and accelerate the migration of data about the company’s network into an inventory system.
Contents:
Industry: Telecommunications
Country: Poland
Founded in 2017, Nexera is the first large-scale wholesale telecommunications operator in Poland. The company builds a high bandwidth, fiber-based next-generation access network (NGA). The operator takes part in the Operational Programme Digital Poland, building a fiber optic network around the central and northeastern parts of the country. The aim is to provide access to over 1 million households and over 3 thousand schools and other educational units.
Challenges
Being a modern telco operator, Nexera implemented a network inventory system to manage their business processes in a digital form. The key element of the inventory system implementation was moving data about the network based on the project (as-built) documentation in CAD format. The company completed the data migration into the Cross inventory system.
In the initial stage, it turned out that the data in the project documentation contained many errors such as unclear connections between network points or a lack of information within the attributes. This greatly complicated the migration process. Anytime the system would encounter a problem, the entire process was suspended. This required identifying and finding the error, fixing it, and resuming the data migration. The need for repeating particular steps made the migration overly laborious and time-consuming.
Therefore, Nexera needed a solution that would allow them to complete an effective data validation and repair. This would help the operator to promptly and seamlessly migrate project documentation into the inventory system.
The scale of the project
15,500
The area of the inventoried network
13,000
The total length of the fiber optic network
92,000
The number of the fiber optic connections
168,000
The number of the other network objects (e.g. wells, poles)
6,200
The length of the technical cable line infrastructure
Solution
Nexera reached out to Globema, the only distributor of the FME software in Central and Eastern Europe. FME is used for data processing, integration, and migration, and offers exceptional support for geospatial data.
Globema’s task was to help the operator process data about close to 13k kilometers of fiber optic network and over 6k kilometers of cable line technical infrastructure. Those data were then to be loaded into Nexera’s inventory system.
Globema, together with the as-built documentation producer, used an FME script for creating input files that are loaded into the inventory system at the right moment. Simultaneously, we created a mechanism that automatically identifies and repairs errors in the documentation.
FME allowed the team to detect and fix errors before data were to be loaded into the system. This limited the number of repeats needed for loading migrated data and accelerated the whole process.
Analysis of source files – preparing for validation
Before starting the actual validation process, we had to analyze the source data in detail:
- DWG files containing information about the network’s technical architecture (e.g. wells, poles, cables, splices) and the address point location.
- XLSX files present complete and detailed data about the network and address point attributes.
We merged data into a single set of input files (files processed so that they could be loaded into Nexera’s network inventory system). In the next step, we cleared the discrepancies that appeared after the data integration. We converted the cleaned data into a single, defined CSV format, so they could be loaded into the network inventory system.
The FME script for input file analysis
Data validation and migration using FME
The validation stage includes verifying spatial relations between linear and point objects within the network, such as cables, wells, poles, splices, etc. The attribute values are also analyzed, as it is needed to verify whether they are consistent with the lexical data. Validation covers the network consistency as well – this is done by e.g. finding paths between any two nodes within the network.
During the validation, errors get fixed separately for the geometry and for attributes:
- Geometry errors: unclear connections between network points, wrong object locations, or topological errors
- Attributes errors: lack of information in the attributes or their values or a need for adjusting to lexical values, not only one-to-one but also through an analysis of the text similarities.
The system designed by Globema uses FME and automatically identifies and fixes errors. It also generates reports about errors necessary to fix before moving data into the inventory system. This significantly decreases the number of data migration repeats.
A fragment of an FME script for data validation
Stages of the data validation process:
- Automatically selecting files for validation and repair / Downloading data from the input repository
- Identifying errors in the original documentation
- Processing loaded data and automatically fixing errors
- Saving report files and generating statistics of the entire process
Data validation and migration diagram
Using FME in the process allowed us to automate some processes:
- Downloading data from the input repository (1)
- Validating data (2)
- Fixing errors, preparing reports about corrected data and remaining errors that require further, manual repairs (3)
- Sending data and reports to the report repository (4)
- Downloading information about errors fixed by FME from the FTP server (7)
- Validating and fixing some of the data using FME
Some of the steps still need to be completed manually:
- Classifying errors for repairs (Globema) (5)
- Fixing errors (the network contractor) (6)
- Sending the report about fixed errors to the FTP server (the network contractor) (7)
The process of validating data, downloading the error report, and fixing errors is performed independently for each network fragment and repeated until every error in the given set gets fixed. When there are no more errors in the documentation, files are ready to be migrated to the inventory system (step number 8 on the diagram).
Data validation reports
During the data validation process, reports are generated for every object type in the DWG format. Those reports define the error type as well as the original and fixed value. Object IDs allow users to find objects in the source data, which are data before validation and the error repair. A user that gets an error report knows, which data need to be corrected and whether they require verifying information out in the field or manually entering changes into a file.
The process is repeated until achieving the correct documentation free from any errors. Then it can be entered into the inventory system.
Data validation report in FME
Designing the process for verifying project documentation quality
The process for automated data verification and error repair, besides the FME Platform, also used the Jira program for project management as well as data catalogs on the FTP server.
1
FTP
Here, input data are placed in corresponding catalogs and can be used further in a project. Verified and corrected data as well as error reports are also sent directly from FME Server. Folders are created automatically based on input file names.
2
FME Server
The program that automatically scans data, searches for errors, and fixes most of them. Some of the errors cannot be fixed automatically as they might require manual file modifications or sending a technician to check the network in the field.
3
Jira
The system used by our client for task management that automatically detects changes in the FTP server (such as new input files of network documentation, error reports, etc.) and creates tasks to complete.
After an appropriate configuration, the tool may be shared further with the next users, including the network executors. This improves the workflow tremendously, as Nexera doesn’t need to step in and load source files into the validation system.
See where we use AI/ML!
Globema completes projects for telecommunications and network operators, including collecting and maintaining network data. These projects are based on a set of tools and services called GlobIQ.
Besides migrating data from digital sources, we also retrieve information from paper documentation using AI/ML methods. We also support data collection in the field with specialized mobile apps.
Get to know the GlobIQ platform
Benefits
With Globema’s solution, Nexera has significantly accelerated and streamlined the process of migrating telecom network data to the network inventory system. The use of FME allowed automatic data inspection and error repair even before the migration began, thus reducing the number of iterations of the data loading process and increasing the quality of the input data.
As a result:
Thanks to FME, the mechanism developed by Globema enabled Nexera to expedite the migration of network data into our inventory system, facilitating the rapid deployment of high-quality internet services in line with the Digital Poland Operational Programme.
Jacek Wiśniewski, Chief Executive Officer, Nexera
Would you like to achieve similar results?
Check if this solution would work well for your company. Fill out the form and our consultant will reach out to you to discuss your needs and schedule a demo.
fme@globema.pl
+ 48 22 848 73 13
Read other customer stories
Grupa MTP
Saving time and reducing expenses through integrating different systems and data sources
Learn how Grupa MTP achieved efficient information flow between different departments and customers using a data information platform.
Visoro Group
Automating manual processes and sending data using a single script
Learn how Visoro Group used the FME Platform to automate and speed up data processing and integration by 750 times.