Removed upsert identifiers in S3 also get unmapped

  • 1
  • Problem
  • Updated 2 weeks ago
I think I just discovered either a bug or something that at the very least needs to be made much clearer than it currently is. 

A customer of mine changed the configuration of one of their S3 jobs, which led to us removing two of the three identifiers in the upsert configuration. The issue is that removing those two upsert identifiers also caused those two fields to be unmapped in the field mapping, even though we are still mapping those two fields in the job. This caused the job to fail and for us to troubleshoot and figure out the root cause. 

Seems odd to automatically unmap those fields in this scenario.
Photo of Spencer Engel

Spencer Engel, Employee

  • 5,814 Points 5k badge 2x thumb

Posted 3 weeks ago

  • 1
Photo of Parthiban Periasami

Parthiban Periasami, Employee

  • 272 Points 250 badge 2x thumb
Hi Spencer Engel
The logic in implementation is if upsert key is added, add the same field automation in mapping section. But, while removing the upsert field need not remove from mapping section. I will talk to team on this behaviour.