Today I had the first occasion to load data to an MDA via CSV upload since Data Management’s Horizon-ification update. It has not been pleasant. I don’t mean to come in too hot here but I did want to share my experience and some (hopefully constructive) suggestions/critique. Below are some of the issues I had and accompanying suggestions.
1.) When creating the schema for a new MDA via CSV upload, all fields defaulted to Boolean datatypes. This may have been an oversight on my part, but I don’t recall any obvious option to configure that during the upload.
If a default datatype needs to be selected as a starting point, can I suggest that it at least be String? Boolean feels like it will be incorrect more often than it happens to be correct - at least String has a fighting chance.
2.) When loading actual data, the Data Type Attribute column is entirely blank…
3.) After verifying fields mappings and hitting Upload CSV, nothing much happens. A progress spinner can be seen behind the field mapping window. It is not obvious what, if anything, is happening at this point. Can I close out of the field mapping window, or will that interrupt the upload?
This is where I stopped understanding whether my clicks had been acknowledged, and what was happening with the data. At minimum, that progress spinner should be brought to the foreground in front of the field mapping window and it should be paired with a short message explaining that data is being processed. The user should not be able to interact with field mapping config or hit the upload/cancel buttons at this point.
4.) Eventually, the progress spinner vanishes but I am still hanging out on the field mapping window. Do I close it now?
A message explaining the outcome of the action should pop up here once it is finished, OR if it will take a while that should at least be explained. A success notification email should be sent out once the process finishes. The field mapping window should close, and the user should be returned to (an automatically refreshed) Data Operations page.
5.) Ok I closed the window, but there are no records in the MDA according to the Data Operations screen. Using the refresh button on the screen does nothing. Refreshing the entire webpage and then finding and reselecting the MDA, then navigating back to the Data Operations tab, DOES show some data - but far fewer records than I’d expected.
It was not clear that data was accepted at this point, if it was still being processed, etc. The built-in refresh button may need some work as well, since I had to refresh the entire browser window to see any new data. That may have been coincidental - data may have loaded while I was refreshing the browser and navigating back to the MDA to check again.
6.) I decide to delete the partial dataset so I can start over, because I don’t believe I’m able to specify an upsert key when uploading data from a CSV here - so I’ll need to clean out the existing data before retrying to avoid duplicate records.
Am I correct that you cannot upsert during a direct upload via CSV? That should be an option. If it IS an option, I completely missed it.
7.) The deletion process is super slow. During this process, the missing records start to trickle in after a few refreshes - so I guess it wasn’t done loading the data. Since I’ve already started deleting the data, though, I have to now wait for the deletions to finish AND for the missing data-in-transit to load to the MDA. I have no indication of how that process is proceeding.
It feels like record deletion is MUCH slower than it used to be. Is there a reason for that? I got a series of 3 emails reporting the results of the data deletion, two of which told me that the actual number of records deleted was fewer than the expected number - with no other information as to why.
Final thoughts…
The experience of setting the schema for a new MDA via CSV upload, and especially the experience of attempting to upload data to that MDA via CSV, is not very intuitive or responsive. I think a small acknowledgement after users take an action would go a long way towards improving the feel of the process, but overall there seem to be some performance issues that I don’t recall having in the old system.
I appreciate that you guys probably worked hard on this, and all, product updates and I don’t want to sound too harsh. I appreciate the work you guys put in, but this particular update needs work if it’s going to be anything I would willingly use for my work again. I will be running CSV uploads via the rules engine instead.
+1, the new UI makes it cumbersome to upload CSVs to create and add records to custom MDA objects. For example, I can no longer see the full field and column names when trying to map fields and I sometimes run into issues where it seems to “complete” but records are not actually loaded.
I’ve just had another go at this and had another note.
When creating fields via a csv upload I found that not only are the default datatypes set to boolean (covered in my original post) but the process of updating a large number of field datatypes is REALLY painful for two reason:
1.) Every time you click the datatype field to change it, the details pane is expanded for the field you’re editing and you have to either recollapse it or scroll down. I am adding 147 fields. Big sighs.
2.) After finishing all of that up it seems that I have to click Save a bunch - possibly as much as once per every new field? I don’t know but it was certainly more than once and the screen kept jumping around the list of new fields at random so I assume it was a Save specific to each item. This was really not very much fun x147.
Sorry another thing. Resolving timezones on DateTime fields does this for TZ’s with multiple localizations within them...
Also, can we talk about losing the ability to upload a CSV to the company object in Data Management? Another CSV improvement!@anirbandutta any movement on this?