What is the long term strategy/roadmap around dataset creation?

  • 23 July 2020
  • 6 replies

Userlevel 6
Badge +2

I feel like many posts throughout the community have shown a significant interest in having a ‘query library’ in that you could create datasets in a single location and then reference them throughout the platform.

This has been highlighted as a need in Program development, having to create all Queries in Rules Engine for troubleshooting, and then building a second time in Programs (which has since been solved for with execution history but it’s duplicative if an existing Rule is exactly what is needed in a Program).

It’s been highlighted as a need within Programs and the ability to duplicate a query as a second participant source within a program.

It’s been highlighted as a need for reporting. 


Data Designer -sort of- solves for this - but it’s not in replacement of, it’s in addition to. And now we have Segments. So you can reference Data Spaces, Rules Engine, Data Designs, Query Builder, and Segments. For Programs, data designer present the issue of the design refreshing independent of the program. This week, I couldn’t figure out why my data design wasn’t coming together and had to rebuild it in Rules Engine ( :face_palm_tone2: ), where I find it’s easier to troubleshoot.

Long term - is there any intent for one solution to replace them all, so there is a single source to compile and manipulate data?

6 replies

Userlevel 7
Badge +3

Very interested in understanding this as well.  Is there a plan to leverage a core functional method across all these features?

Userlevel 7
Badge +2

Big-time +1 to this

Userlevel 1

This would be amazing and make life so much easier! +1

Userlevel 6
Badge +1

Hi All,


In the long term, we are thinking of some sort of convergence here. I can share more details once the proposals are finalized. The objective is to have a consistent and single data prep experience powering multiple products. At the same time bring some amount of reusability of data sets/designs across products.

Userlevel 6
Badge +2

Is there an update on this @rakesh ? There are still so many scenarios where the same query criteria is used for a program and a rule with minor adjustments to filtering. To have to build two programs and two rules which are almost identical is still so time consuming.

It seems like data design is where it seems to be going but effectively creating a custom object for something that is a one-time use doesn’t seem to be a prudent use of system resources either.

Userlevel 6
Badge +1

Hi @kelly 

Thanks for bringing this up. 

We have 2 different ways to re-use info across systems. 

  1. Data level - this is what you have just described and you can achieve this today. 
  2. Meta data / Query level -  In Data Designer (or other products that have Data Designer UI), you can save your steps as “Design Templates”. These steps can then be created with one drag and drop in any of the products that have Data Designer UI. Today you can do this with Rules engine. In the future, we plan to add this to JO as well.