I am looking for an Skype/Teams conversations with some Expert on ETL tools (SSIS, Talend, Knime, etc) to help me understand which is the right tool that I need to implement in my job needs. You need to show me some small example as well.
I need help to solve the following problems and some knowledge advice sharing with me:
-I need to create reports where I get data from multiple sources (files saved in a folder, SQL Server, etc) then i need to do some ETL things (renaming headers, emptying empty rows, adding extra columns, calculated columns, etc) then I need to do some relationship between different tables coming from different sources. After I need to create a final report in an a table or excel file as output. After this final report I need to automate and send to the users and at the same time I need to store the data in SQL Server as well. These should all be designed once, deployed and automated. So that we do not need to do any manual work.
How to do ETL? How to load data to SQL Server? How to flow the data from Excel files to SQL Server. How to schedule those data flows from Source to Destination?
The idea is that do once all the process and flow the data between different systems. Say from (Excel files, folder, Oracle serer to SQL Server for example).
You should bring knowledge on: -Very important: How to flow the data from SOURCE TO DESTINATION (Flat file Excel files, etc to SQL Server etc) -Knowledge on Apache Kafka, Airflow, Nifi, -Knowledge on Talend, KNIME -You're expert on ETL tools such as SSIS, Power Bi Query. -Know the BEST ETL for my needs upon our conversation. -Must have extensive knowledge on ETL and Data Integration, Pushing and Pulling. -Must know how much resources is required each software, which is easier, better for moving several thousands of data to flow (It's not about BIG DATA). We work on just several thousands of data. 1K to 500K rows of data each time.
There is no real work. Just talk with me on Skype, show some example, help me to understand and solve my doubts, give me idea and advice on best software for my problem.
We should talk all in 1:30h and I should see some small examples and demos from your side.
