header
Solved

Recurring NPS Surveys

  • 26 February 2020
  • 5 replies
  • 179 views

Userlevel 4
Badge
  • Veteran Rookie
  • 28 replies

Hi folks, I’m looking for a best practice on how to best segment and repeat an NPS survey.

Requirements:

  • I will have numerous audience filters to apply this appropriately to the right users at the right time (i.e. have been a user > 6 months, have visited part of the product XXX times, are currently performing YYY action).
  • Ask this question every 6 months.
  • Ask to 25% of my user base.
  • Do not re-ask same 25% until 100% has been completed.
  • Repeat…

I could set up intervals to make sure it happens once every 6 months:

 

The problem is with segmenting by % of users.  I can do 25% here, but I’m guessing it picks 25% at run time and never changes.

 

I could pick a range here but 1) I don’t know how big my segment is without manually figuring it out, and 2) this requires manual effort to move to another set of users every interval.

 

Really, I’d love to just set up some recurring NPS surveys that don’t pester the same users repeatedly yet continually get feedback without having to re-set-up everything every interval of time.

Understand we’ll be doing product-area-based NPS surveys.  So, we’ll have a LOT of these running.  Manual is not ideal.

 

Appreciate any thoughts.

icon

Best answer by sai_ram 27 April 2020, 13:37

 

Hi folks, I’m looking for a best practice on how to best segment and repeat an NPS survey.

Requirements:

  • I will have numerous audience filters to apply this appropriately to the right users at the right time (i.e. have been a user > 6 months, have visited part of the product XXX times, are currently performing YYY action).
  • Ask this question every 6 months.
  • Ask to 25% of my user base.
  • Do not re-ask same 25% until 100% has been completed.
  • Repeat…

I could set up intervals to make sure it happens once every 6 months:

 

The problem is with segmenting by % of users.  I can do 25% here, but I’m guessing it picks 25% at run time and never changes.

 

I could pick a range here but 1) I don’t know how big my segment is without manually figuring it out, and 2) this requires manual effort to move to another set of users every interval.

 

Really, I’d love to just set up some recurring NPS surveys that don’t pester the same users repeatedly yet continually get feedback without having to re-set-up everything every interval of time.

Understand we’ll be doing product-area-based NPS surveys.  So, we’ll have a LOT of these running.  Manual is not ideal.

 

Appreciate any thoughts.

 

 @alb  Configure Recurring Interval for Surveys is possible now.

To configure this enhancement for Slider, Guide, or Survey engagement:

  1. Create a Slider/Guide or a Survey engagement.
  2. Navigate to the Schedule section.
  3. Select Intervals under the Qualification Scope section.

For more information please go through this article

 

View original

5 replies

Userlevel 4
Badge

Hi Al,

 

How about targeting all the users with configuring the audience logic of triggering the NPS engagement only if the NPS is completed a number of times is zero(“0”)

 

  • I will have numerous audience filters to apply this appropriately to the right users at the right time (i.e. have been a user > 6 months, have visited part of the product XXX times, are currently performing YYY action).
  • Ask this question every 6 months. -- You can use the qualification scope.
  • Ask to 25% of my user base. -- I think you can target all 100% with the audience configuration
  • Do not re-ask the same 25% until 100% has been completed. -- Can filter them based on the audience logic as shown below

 

If the above don’t meet your use case, please feel free to share a bit more detail, so that we can serve you better.

Userlevel 4
Badge

Hi Dileep, not sure I follow your response nor am I sure you understood my question.  Let me elaborate a bit more.

NPS is something that never ends.  You should run these surveys continuously to see trends and changes over time.  I want to set them up, and maybe never have to go back into them again.

  1. I want each user to only receive the same survey once within a window of time, then I would like it to repeat...and keep repeating.
  2. I want the survey to be spread out over that window of time.  Meaning, don’t ask everyone all at the same time.

So if I were to map out an actual example, let’s say:

  • I have 100,000 users in a particular Audience segment.
  • I want to ask each of these users their NPS score one time every 12 months.
  • I would like this survey to be spread out across these users on a monthly basis.  So, 8333 users (100,000 users / 12 mo) would receive this survey in month 1.  A different 8333 would receive it in month 2, and so on.  Once completed, the cycle would reset and start again after the 12th month.
    (Note that I may want to do this on a weekly, daily, hourly, quarterly...whatever basis.)

The point again is to set this up and let it fly.  Potentially forever.  With this model there are always users providing NPS input at any given time, so you can even do cohort analysis on this data to see if they’re happier after products have been released or improved.

 

Userlevel 5

Hi Al,

 

I would like to understand the reason behind wanting to spread it out based on the % of users?

 

Also, just to clarify, by the setting the Audience Criteria + Qualification scope (QS), the system any way spreads it out. In you above example, you may hypothetically assume that if your QS is Once every 6 months, all your NPS Responses will be collected around the same time period but that’s not fully true as the Audience Criteria comes into play at this point.

I am sure you will have a filtered set of users through Audience Criteria, for eg users who:

  • -  signed up >30 days​​​​​​​
  • visited > 5 or xx times
  • Used Features a, b, b > xx times
  • URL Page: abc.xyc.com

So, the NPS will be triggered each time only to users to satisfy the above criteria and over the course of 6 months, there will be new users who keep qualifying and get this NPS.
Only the first time that you launch, you may see a huge influx but you will see spread out responses over 6 months.

For the first time when you launch, if you want to avoid the large influx around the same time period, you can use the “Engage xx % of the Audience” and then manually/ gradually increase it in the span of 6 months, again recommend doing this only initially as you launch.
 

As for being able to correlate the NPS Sentiment after a Product Release, it is not a recommended best practice as NPS is not tied tied ‘only’ to a Product Release and it may not be the best factor to correlate to. For Product Releases, the best practice is to capture Release Feedback typically after 2-4 weeks of the release.

However, you can still set your Qualification Scope to Quarterly, Monthly, every 6 months and put it around the Product Release Schedule.

Also, a better way to correlate customer sentiment to the Product Release might be by looking at the Adoption Report of the newly released feature and analysis the NPS Sentiment of those users over a period of time.

Again, I’d like to re-emphasize that as part of NPS Sentiment Analysis, it is best to keep it separate from the Product Release Feedback as there are multiple factors that play into the customer sentiment as is typically meant towards “How likely are you to recommend Company ABC” (as you may already know).

Hope this helps as the product currently still allows you to achieve what you are trying to do (not in the most automated way which on purpose has not been designed that way yet) and also, would be nice to revisit the approach to this use case.

Userlevel 4
Badge

Hi Harshi, thanks for the reply.  We talked about most of this on the phone today, but I’ll reiterate here.

  1. This request applies to ALL survey types.  NPS, CES, Star, etc. 
  2. I realize NPS and product release aren’t an exact corollary, just making a point for why I want to spread out users and not get a one-time bomb when first launched.  The point is to get a constant steady stream of feedback so you can make correlations to product changes or any other changes.  You changed a workflow and the CES for that workflow increased.  Awesome.  Shouldn’t have to set up a new CES just to find that out.
  3.  I realize I can manually change these things on a cadence, but PX should be teaching customers that feedback isn’t a one-time thing that you should have to babysit.  PX should be teaching that you should be querying your users constantly over time so you always have data to make decisions with.  If PX wants to tell that story, then PX should make that easy!  <smile>

Cheers,

Al

Userlevel 7
Badge

 

Hi folks, I’m looking for a best practice on how to best segment and repeat an NPS survey.

Requirements:

  • I will have numerous audience filters to apply this appropriately to the right users at the right time (i.e. have been a user > 6 months, have visited part of the product XXX times, are currently performing YYY action).
  • Ask this question every 6 months.
  • Ask to 25% of my user base.
  • Do not re-ask same 25% until 100% has been completed.
  • Repeat…

I could set up intervals to make sure it happens once every 6 months:

 

The problem is with segmenting by % of users.  I can do 25% here, but I’m guessing it picks 25% at run time and never changes.

 

I could pick a range here but 1) I don’t know how big my segment is without manually figuring it out, and 2) this requires manual effort to move to another set of users every interval.

 

Really, I’d love to just set up some recurring NPS surveys that don’t pester the same users repeatedly yet continually get feedback without having to re-set-up everything every interval of time.

Understand we’ll be doing product-area-based NPS surveys.  So, we’ll have a LOT of these running.  Manual is not ideal.

 

Appreciate any thoughts.

 

 @alb  Configure Recurring Interval for Surveys is possible now.

To configure this enhancement for Slider, Guide, or Survey engagement:

  1. Create a Slider/Guide or a Survey engagement.
  2. Navigate to the Schedule section.
  3. Select Intervals under the Qualification Scope section.

For more information please go through this article

 

Reply