Reengineering of ETL Process from Perl into J2EE!
“We were satisfied with the well going of the development process ensured by a brilliant combination of technologies and good communication skills of the team”
The end customer requested the migration of an existing ETL application from Perl technology to J2EE. The final purpose was to obtain a future-proof application based on standardized technologies.
The ETL application is used by a hardware vendor in order to process sales data from hardware resellers and it is integrated in the producer’s CRM software.
Solution by Ropardo
The application needed to be built on J2EE technology using Vignette Portal as front end GUI and Oracle Database.
It had as main functionality data extraction and transfer to one of two customer proprietary formats. The data entry formats were ‘comma separated values’ or ‘tab separated values’ exported from MS Excel.
A custom data load and translation module and an archiving component were built to store data in an Oracle 10g database.
The application had to conform to JSR Portlets specification and run on the UNIX platform.
After the development phase, Ropardo has delivered:
- component integration in the portal in .war format
- he web component in .war format
- the stand alone backend component in .jar format
- instructions for database, portal and backend installing and configuration
- quality assurance process
Ropardo team ensured the study and reengineering process of the existing application and definition of requirements, defining the application architecture, software development and testing (Quality Assurance).
JSP was the view technology used for user interaction within the portal and web application.
Spring was used as J2EE MVC application framework to provide centralized, automated configuration and wiring of the application components: clean assembly of used view technology (JSP) (presentation layer), transformation logic (business logic layer), database access (persistence layer).
The application was integrated with WebLogic 9 application server and Vignette Portal.
Data was loaded to an Oracle 10g data warehouse.
JDBC and Hibernate were used for ORM (Object Relational Mapping) integration with Oracle database.
Log4J was used for application logging.
Ant was used for build process automation.
Junit was used for unit testing.
Quartz was used as the scheduling engine in the testing phase for timing and job management.
PMD was used for source code quality assurance.
SVN was used for code management
Experience & technical expertise
Ropardo already had prequalified human resources which are skilled in the particular IT domain that project required. They have worked on a large number of projects in the area, thus gaining long-time experience, which allows them to avoid downtime, since they have already encountered specific domain problems, have successfully dealt with them and have the answers ready.
Reduced costs & shorter development time
Since prequalified people were hired, MGM-TP avoided the costs resulting from training their own team. The off-shore solution also allowed MGM-TP to save the time they would have normally invested in the training of the team.