Many companies have faced the situation when the document management system, which was faithful and true, becomes obsolete, accumulates a lot of uncorrected mistakes, rests on the ceiling of its development. In this case, the owners of the system realize the need to replace it.
The company NAUKA shared their experience of complete replacement of the ECM and electronic archive of documents at the conference “DMS and ECM Day 2018”. Later this experience helped us a lot in the implementation of other similar projects. We are publishing this case for you.
Our customer was a large oil refinery with a lot of automated processes, both production and administrative. To manage documents and organize an electronic archive they used EMC Documentum that was deeply integrated into the IT infrastructure of the plant. The platform provided an access to document management for a variety of information systems. In addition, an electronic document management system was deployed on its basis.
The platform served two repositories – for the EDS and for the electronic archive – with different approaches to differentiation of access rights and authentication. To store content and metadata, we used file storage on the storage system and the MS SQL database, respectively.
NAUKA had been accompanying and developing all this process. We had finalized the American system at the request of the customer, improved the interaction of information systems. But one day, both our customer and we, realized that there is less and less room for development of the system. The number of workarounds that were being used to overcome the limitations of the current version of the platform was growing like a snowball. This inevitably led to a decrease in productivity and made the platform not so easy to use.
The issue of updating the system ripened. However, it was precisely at that time that the crisis hit the country, exchange rates soared, and the media was full of the word «sanctions».
One of the most painful problems for business became the cost. Licensed versions of the foreign system were too expensive.
It was then that the plant made a difficult decision to replace the SED and the electronic archive of documents with a platform by another manufacturer.
Requirements for the new platform were formulated. It was required to inherit all the existing Documentum functionality, including highly reworked ECM components, integration with other systems – SAP, ArcGIS, the information portal, ERP modules, etc. Some of the functionality needed to be optimized, in particular, we needed to:
- spread audit onto different tables,
- combine two repositories into one with common authentication,
- use the Oracle database instead of MS SQL,
- the file storage with a capacity of 2 TB needed to be transferred to another storage server.
Одна из самых сложных задач: исполнитель должен был обеспечить перенос всех данных на новою платформу.
One of the most difficult tasks was to ensure the transfer of all the data to the new platform.
Unfortunately, a suitable solution was not found at that time.
The customer paused, and we began to think how we could help in that situation. Looking back on our 12-year experience in creating information systems and 9-year experience in supporting third-party ECM, we realized that we could create our own platform. On the one hand, it had to be versatile and flexible enough to meet all current and future customer’s requirements, and on the other hand, it had to be free of the shortcomings of the foreign system.
Despite the scale of the project, the platform was created within the shortest possible time and offered to the plant. Having successfully passed the compliance test, it was accepted by the customer as a complete replacement of the obsolete platform.
It seemed that the most difficult thing – the development and testing of the new system – was over. All that remained to be done before we could get down to work was to transfer the existing data. We enthusiastically began to compile a list of data that needed to be transferred. 2,000,000 documents, 3,000,000 audit records, 2,000 users, electronic tasks, communications, access rights sets… It seemed that the list would never end.
In addition, the new architecture of the system, the new database required a different approach to the location of the data. Just transferring them “as is” would be wrong as they can be changed during the transfer process and, therefore, must follow strictly defined routes, depending on the type of data and its properties.
To solve this problem, a multithreaded migrator program was created, which:
- received the data from several sources;
- identified data type;
- distributed the data along the routes, according to the required logic;
- ensured the integrity of the transferred data;
- recorded data in the new database.
The program allowed to completely automate the data transfer process. And we were able to proceed with the complex and demanding process of migration with pinpoint accuracy.
The process was started and then tuned, but we should not forget that the refinery has a continuous production cycle. Most of the technological processes do not stop for a minute. Therefore, we were tasked with carrying out the migration within the shortest possible time. For example, we had only two days to completely replace the ECM. How did we deal with that huge array of data?
First, we used two platforms at the same time. Components were transferred one by one, not immediately. Some modules continued to work on the old platform, others – on the new one. This required additional resources and work to ensure data consistency, but it allowed to transfer a huge number of ERP modules smoothly to the new platform without interruptions in work.
Second, due to our deep knowledge of the American system and reverse engineering, we understood the scheme of the Documentum database and were able to transfer data directly from one database to the other. Thus, the speed of migration was limited only by the technical capabilities of the databases themselves. To speed up the process, the migrator program shared the data into several threads.
Third, since there was a lot of data that was being transferred quickly, it was necessary to check its correctness. For this purpose, a separate module was developed, which provided a constant check of the correctness and completeness of the transferred data. Manual testing was carried out both on our side and on the customer’s side. Thanks to this module, there were no problems with data consistency.
Next, in order not to waste time and money retraining users, we ensured continuity of the interface. We kept the locations of buttons and fields, did not change anything in the usual business processes – in general, everything remained in its place.
As a result, over two days off the entire electronic document management system of the huge enterprise moved to a new platform – even those who missed the message about the transition or did not know about it, were only slightly surprised by the updated design and quietly continued to work. The rest of the integrated information systems did not stop at all during the “move”.
Finally, the outdated platform based on EMC Documentum was completely replaced by a promising platform of its own development, almost imperceptibly for users of information systems.
The customer received
- the system that fully meets its requirements, which he can develop further on the basis of open-source solutions;
- migration of absolutely all data – the entire history of documents, the entire audit, all tasks and communications remained in their original form;
- optimization of the system – increased speed;
- replacing the problematic UCF with a separate file agent that has successfully proven itself;
- simplified ability to customize the system in the future.
All this work led to the creation of its own ECM NAUKA. The platform has already experienced three successful implementations, comparable in scale and complexity. Our ECM turned out really universal, compatible with a lot of databases, flexible, functional and affordable. It has everything you need for a modern enterprise. Leave your application in the form of feedback and see the advantages of ECM NAUKA!