Solved

How to remove duplicate values from Union in Data Designer or from Rule?any suggestion?

  • 10 July 2022
  • 4 replies
  • 223 views

Combined 2 tables in Data Designer via Union but gives me duplicate rows.How should i remove them?

icon

Best answer by Wayne 26 August 2022, 02:15

View original

4 replies

Userlevel 6
Badge +1

@Syed 

The best possible path is to avoid duplicates when you do a union itself. Make sure you use the right filters so that a single record doesnt come in both datasets. If there is any usecase that makes this impossible, could you please post that here? 

 

That being said, one way to remove duplicates is to load the data to an(other) object and use correct identifiers. 

Hi @rakesh 

Below are as an example 2 samples. when i union them.It gives me 4 rows.what filter i should add that

it should not  show rows which have not values in Serial 1, Serial 2, serial 3 andIf any row has value iether of these fields then it should show.In this example Table 1 should show ID 2 and Table 2 should show ID 1

 

@Syed - I believe I’m having the same issue as you. I was using Union to combine based on Advanced Outreach Name but it’s splitting up both like so. Is that correct?

 

 

Userlevel 5
Badge

You could probably get away with just applying a filter where Serial 1 != null AND Serial 2 != null AND Serial 3 != null AND Serial 4 != null post this union. That would remove those blank rows. 

 

Unless it’s possible for an ID to be blank in both data sets and you want to reflect that. If that is the case then you will need a different solution.

 

Merging instead of union and then case statement your duplicate fields to combine them. If Table 1.Serial 1 != null then output Table 1.Serial 1 Else Output Table 2.Serial 1

Reply