Under Consideration

S3 Connector file ingest without header row

  • 23 November 2016
  • 7 replies
  • 42 views

Userlevel 5
Badge
The field mapping for an S3 Connector file ingest currently requires the first row of the csv to be a header row with the column name for mapping.  It would be great if there was an option to uncheck header row and just map on column number.

Use Case: From our data warehouse we have to write custom SQL for every unload to S3 in order to add this header row.  Our BI tool allows for scheduling report output to S3 but does not include a header row. Maintenance would be drastically reduced if had this functionality from Gainsight so that we could manage data pushes from a report instead of custom SQL.

7 replies

Userlevel 4
Hi,

Thanks for posting your idea on this!! 

We did receive same request a few months ago and at that time, we did not consider this flexibility because data Integration mapping is tightly bound to the order of the columns in the file. While generating the file in your DW, if the order of the column is changed, It is hard to diagnose the data quality issues!

While I can't give you an ETA, we will certainly revisit this flexibility (may be with some caveats like order + data type match ) as part of our S3 connector V2, which is in our roadmap.  
Userlevel 7
Badge
Ashok, I also have a customer that would like to do this. S3 or API would be fine. Similarly, it comes down to not having the ability to edit the file upon export from their BI tool. Going off of column position in the file would probably work.
Userlevel 6
Badge
+1 for this idea! This is a huge setback for integrating data from an already custom built report and has become a blocker. Could this make it into the spring release?
Userlevel 6
Badge
Where is the S3 connector V2 currently on your roadmap? This is a must need for my organization.
Userlevel 4
Hi Joseph and Renee,

S3 file ingest without a header row is on our roadmap items list 
where we would have to options:
a)File has headers
b)File does not have headers(order of columns will be considered)

Will keep you posted on the ETA and try to deliver this in spring

Thanks and Regards,
Lakshmi
Userlevel 6
Badge
Lakshmi - We would need to load the data from the S3 in to predefined columns in the MDA--unless Joe calls me a fibber 🙂 --but if this could be done--and during the
Spring release--it would mean a world of difference for my organization. Please keep me updated.
Userlevel 4
Hi Renee,

Sure! Will keep you updated on the progress

Thanks and Regards, 
Lakshmi

Reply