BJ’s Wholesale Club

Creating a tailored shopping experience based on fulfillment preferences

 
 
 
 

Tackling a cart abandonment issue from the top of the funnel through fulfillment selection.

 

Role

UX Lead

Activities

Unmoderated user testing - quantitative surveys - user journey maps - information architecture - wireframes - moderated usability testing - LogRocket analysis

 
 

The Challenge

Data showed that on BJ’s.com and the BJ’s app, mixed carts (those with items divided between pickup, delivery and/or shipping) had an abandonment rate that was 25% more than carts with a single fulfillment method. We were tasked with solving for this discrepancy.

The Process

Diving into bulk edit

This data came to our attention during a cart redesign project, so when various stakeholders were informed of it, they were eager to get a solution built as fast as possible, and one that we could naturally bake into the overarching cart project. Without conducting any user testing, and only focusing on the cart as the issue, the perceived top-down problem statement became:

On BJs.com and the BJ’s app, it is difficult to rearrange items in the cart. This causes problems for users as it is time consuming, frustrating and leads to purchase abandonment. This is important because BJ’s is losing out on 48% of orders due to abandoned mixed carts, and members are less likely to purchase from BJ’s online in the future.

When looking at the issue with this narrow focus, the perceived best solution was to allow users to edit their cart in bulk, vs. item-by-item as it existed. So instead of a user having to manually move 10 items from Delivery to Pickup, one could do it with 2 clicks, saving a lot of time, frustration and, ideally, cart abandonment!

Further defining the problem

Although we were tasked to keep Bulk Edit moving, we weren’t 100% confident that it was right solve, nor were we 100% confident that we were even solving for the right problem. There were many unanswered questions and ignored factors in our initial problem statement – the biggest being, why do so many users have mixed carts in the first place? In order to answer some of these questions, we launched a survey to 20,000 members to better understand their shopping preferences. Our biggest learnings told us that the problem actually lied at the top of the funnel within shopping path:

78%

13%

85%

of respondents begin their BJ’s online shopping trip with a preferred fulfillment method in mind.

of respondents have ever had to change that fulfillment method or add an additional method mid-shopping.

despite the above, 85% of users have had to reconfigure their cart and move all items from one fulfillment method to another.

With these results, it became clear that the vast majority of carts aren’t actually meant to be mixed. Users typically shop with a single fulfillment method in mind when they start shopping, so the issue lies more in the communication and selection of fulfillment throughout the shopping journey.

In our efforts to continue to drill down into the exact areas on site that produce the most issues, we worked closely with out data team to uncover more insights that would help us focus in on problem areas, asking questions like:

  1. Where are users most frequently adding items to their cart?

  2. Where are users most actively selecting and changing their fulfillment methods on site?

  3. How many users are using the fulfillment landing pages to do their shopping?

  4. How many users are using the fulfillment filters on PLP?

  5. What is the breakdown of cart make-up and what are the corresponding abandonment rates?

With these findings, we analyzed the existing experience further and found that:

Most Add to Carts happen on the product listing page (PLP) where fulfillment availability is extremely unclear.

On BJs.com and the BJ’s app, it is difficult to shop within a preferred fulfillment method, resulting in items often being added to cart in the wrong place. This is a problem as users must unexpectedly rearrange items in their cart, a task that is time consuming and manual. This is important because BJ’s is losing out on 48% of orders due to mixed carts, and members are less likely to purchase from BJ’s online in the future.

Although we offer the functionality to filter by fulfillment, and exclusive PLP’s per method, hardly anyone uses them.

Fulfillment upon Add to Cart is essentially hidden, making it easy to add items to the wrong fulfillment methods.

We were able to craft a new problem statement that actually addressed the true root cause of the cart abandonment issue, and one for which we could start doing real discovery around how to fix:

Mapping out discovery

We next put together our primary discovery goal, as well as the objectives and corresponding research tactics that ladder up to it, in order further drill down into how we might solve for this problem on the site and app:

Discovery Goal:

Define an ideal shopping path experience that allows users to easily build their cart in the fulfillment method they prefer.

Research Objective:

  • Understand gap between existing and required fulfillment communication touch points throughout the user journey

  • Understand fulfillment selection behavior

  • Identify optimization opportunities based on member preferences

  • Identify happy paths and edge cases based on member shopping tendencies

Research Tactic:

  • Experience audit, Competitive analysis, Analytics

  • Competitive user test

  • Survey & user testing

  • Survey, user journey mapping

Understanding fulfillment communication touch points

Competitive audit

We reviewed a number of retailer sites that offer multiple fulfillments, to understand all the aspects of their fulfillment selection experiences as well as where they communicated fulfillment availability. The scope of this audit entailed a review of fulfillment selection methods, address and store selector modals, the placement and UI of various feedback and messaging prompts throughout the shopping journey, the inclusion of fulfillment filters and availability on PLP and PDP, and the structure and options in cart in regards to fulfillment selection.

Understanding fulfillment selection behavior

Competitive User Testing

Based on the survey we did earlier in the project, we learned that our members most often shop at competitors, Target and Walmart, two other retailers who also offer multiple fulfillment methods. We crafted a user test to see how users naturally shop with a preferred fulfillment method in mind.

Identifying optimization opportunities

Once we had a better idea of the ideal type of experience for fulfillment selection, both using our user testing results as well as the results from the early quantitative survey, we did a full audit of our current shopping experience to understand all of the areas that needed to be updated.

Mapping out happy paths & edge cases

In our intention to allow users to ‘set and forget’ their fulfillment method, we needed to be sure this wouldn’t leave users stuck in any different scenario. We conducted further research via a second Qualtrics survey to uncover potential edge cases and their ideal solutions….

When shopping your preferred fulfillment method, have you ever purposefully chosen a different method for specific item(s)-- even when those item(s) were available for your preferred method? (ex. you're shopping pickup but purposefully choose to get one of your items for shipping instead)

The majority have not done this, however, almost 20% have. Those who have done this typically say it’s because the item was larger so they needed it shipped vs. pickup. This suggests it is a GM article, which suggests users are likely shopping in these instances more on PDP. Therefore, it is essential we allow users to change their preferred method on PDP.

How often have you changed your pickup club halfway through shopping? / How often have you changed your delivery address halfway through shopping?

The majority have rarely or never done this, showing us that we don’t have to account for the scenario of figuring out availability considerations and corresponding communications for an instance like this. Existing functionality didn’t allow us to separate the order to be fulfilled by multiple clubs, or to multiple addresses, so this was good news that we didn’t have to build it out.

Imagine you set you delivery method to pickup. About halfway through, you change it to delivery -- would you expect the items already in your cart to change to delivery too?

The majority expressed that they would expect this. Unfortunately, this would require the bulk edit functionality, which we learned wouldn’t be possible to undertake for quite sometime. This meant that we needed to ensure proper messaging was in place to set member expectations.

Identifying fulfillment location within the header

Our Shop Your Way project was built concurrently with a new site header redesign, there were synergies between the two projects, so a lot of testing overlapped. A major aspect of both Shop Your Way and the site header was the location of fulfillment selection, as well as what we called it. We used a first-click test to see how fast users could locate fulfillment selection in different areas on the header. We also tested different variations of language to see what most quickly resonated with users and allowed them the most freedom to update different aspects of fulfillment.

Heavy - Address Selection/Confirmation Tool

Heavy - Item Unavailable Modal

Sharing designs with engineering

This part of the project provided us with some valuable lessons in project phasing and iteration. Up until this point we were full steam ahead with a completely reimagined Shop Your Way experience, that didn’t just change site functionality, but also included large scale redesigns of shared modals across the site, including Address & Club Selector modals and Add to Cart success modals. Under the impression that all of this work was in scope and doable for the timeline we had in place, we had a rude awakening when sharing our designs with the head of development. We learned that the robust solution we had fleshed out was not possible with the timeline and resourcing of engineering. It was requested that we scale back our experience to a “Lite” version to be launched ahead of code-freeze. This “lite” approach would also allow us to test and learn in a lower stakes environment.

Scaling back from “Heavy” to “Lite”

Heavy - Site Header

Heavy - Club Selector Tool

Lite - Site Header

Lite - Club Selector Tool

Lite - Address Selection/Confirmation Tool

Lite - Address Selection/Confirmation Tool

Lite - Item Unavailable Modal

The final “heavy” experience

We saw positive progress with the launch of “Lite” so continued onto “Heavy.” Features of the new experience included:

  • Fulfillment selection via the header or first add-to-cart, resulting in the entire site sorting by that fulfillment method as well as subsequent items getting added for the method automatically

  • New modals across the site for add-to-carts, delivery address selection and club selection

  • Fulfillment availability tags across PLP

  • Truncated cart summary

  • Better communication when items are unavailable, fulfillment method is changed or address/club is changed

Happy Path Example

Edge Case Example

Results

  • Add to cart conversion rate increased 5% on web and 12% on app after implementation of fulfillment availability flags

  • Fulfillment selection experience just launched, so too early to tell full performance insights

Learnings & improvements for next time

  1. Going down the path of Bulk Edit before doing any real research was time that would have been better spent assessing the root cause of the problem and working to build a solution that addressed it. Even though Bulk Edit was in theory to be built as part of the cart redesign, by not insisting that we take the problem back and dive into it deeper before starting on a solution, we did ourselves a disservice as a UX Team who should be driving the experience.

  2. Initially, we had no project phasing associated with this update — all aspects (all new modals, new site functionality, product card updates) were set to launch all together at once, driven by the business need to get it released ahead of holiday in hopes of improving sales. This was problematic for 2 reasons:

    • It is better to launch iteratively so we can gather learnings in an agile way and adapt / pivot / make relevant updates / even potentially de-scope updates if iterative updates are solving the problem.

    • Our communication as a team holistically between UX / PM / Developers was not great, and the many, many updates associated with this project were not properly communicated throughout, causing us to have to rework a lot of it to allow for a “lite” version and a “heavy” version based on project scope and engineering resourcing. As a result of this project, we created a new Design to Development handoff process to ensure better communication through a project so that scope creep could be avoided earlier and phasing could also be aligned upon at towards the beginning of the project.

  3. We didn’t put into place any type of post-launch plan to together as a team assess the experience and what iterations and potential shifts were needed.