Background

BJ's is experiencing an increase in Member Care call volume and a decline in member satisfaction. This has prompted us to initiate efforts to enhance BJ’s member communications.

To effectively address these challenges, we have chosen to begin by focusing primarily on email communication, recognizing its significant contribution to these issues.

While our initial emphasis is on email, our future steps will also involve addressing issues and exploring opportunities in SMS and push notification channels to further enhance member satisfaction.

Problem Statement

︎What
BJ’s Members currently receive transactional emails that lack clarity and accuracy, causing member confusion, increased contact volume at member care, and reduced member satisfaction. Our goal is to enhance these communications for a better member experience.


︎Why
We conducted a pre-research survey and found that despite 50% of customers choosing email as their preferred contact channel...



- Pain Points:


- Goals:


︎Who

Road Map

Planned approach for redesigning all touchpoints and establishing member preferences.

Seconday Research

︎Competitive Research
What are other companies doing?
What are the best practices?
Which best practices would work for BJs?



︎Baymard Guideline
To gain a deeper understanding of best practice design guidelines, I conducted research using Baymard as a resource. Here are the key findings:

1. ‘Ensure all order status e-mails have highly visible paths to order actions and account information, and always provide expected arrival dates or date ranges in the order status emails’ (Baymard#878)

2. Information that should be provided in an order confirmation email:

• The order number
  
• The estimated date or date range the items are due to arrive
  
• A shipment tracking link, or information on when users can expect to receive a shipment tracking email
  
• The items that were ordered
  
• The address the item is shipping to
  
• The total cost of the order
  
• The payment method(s), including any applied gift cards or promotional codes
  
• A link to the site where the order was placed with more order details

︎Email Journey Map

I also crafted an email journey map with the goal of gaining insights into the specific use cases that require our attention and enhancing the overall user experience

Primary Research

︎Stakeholders Interview/ Internal Workshop
To gain a deeper understanding of user pain points, we conducted interviews with team members from Membership, Member Care, and Digital Operations. These individuals are invaluable sources of insights because they frequently interact with BJ’s members and have firsthand exposure to the feedback and concerns raised by our members. Here are some of their voices:

Phase 1:
Order Confirmation Email Optimization

We segmented the emails into four categories – 'Order Confirmation emails,' 'Fulfillment emails,' 'Cancellation emails,' and 'Return emails' – based on our analysis of the email journey map. We initiated the redesign with order confirmation emails due to their highest open rate, as BJ's members often rely on them as proof of purchase.

︎Current Emails

Competitor's Content Analysis

︎Some Findings(Order Summary) *The complete Competitor's Content Analysis Report can be found here

Ideation

︎Key Areas of Focus

Brainstorming

︎Self-questioning


When brainstorming, one of the critical questions I pondered was whether we should include item details in our order confirmation emails. I observed that some of our competitors, such as Amazon, Walmart, Instacart, Kroger, Whole Foods, and Stop & Shop, had opted to exclude item details in their order confirmation emails.

Upon closer examination, it became apparent that order confirmation emails couldn't serve as complete 'receipts' due to potential item cancellations and substitutions. From an engineering perspective, displaying extensive data increased the risk of issues. From a business standpoint, directing BJ’s members to the order history page for details had the potential to boost revenue.

Consequently, we made the decision to remove item details from our order confirmation emails


1.  CTA:
‘View or Manage Order’ CTA
‘Add Items To Order’CTA
2.  Q&A
What if I forgot something?
What if something is out of stock?
How will I know when my order is on its way/ready for pickup?
(tires)What if I need to reschedule my tire appointment?
When will I be charged?
Can I cancel my tire order?
3. Your Might Also Like(Shop Deals) 
4. Help menu



When BJ's members open the email, the first thing they see should be the savings on this order.

Prototype

Based on the Research, Ideation, and Brainstorming, I tried to translate the ideas into prototypes.

︎Welcome box

︎Order Details

︎Next Best Actions

︎Bottom Help Menu

User Testing

︎Usability Testing

• Tool: UserTesting.com
• Desktop(6), Mobile(6)
• Open-ended question
  
  

- Scenario
“In this scenario you have already completed your order. You realize you forgot to order an item and wish to add it to your previous order.(Add "dog food" to the order: 1. Search “dog food” 2. Add 2 bags of “Berkley Jensen Deboned Beef” dog food to your order.) Please go through this prototype to complete this task.” 

- Complete Testing Results:
https://drive.google.com/file/d/1ZCo8PcPeWS1BQE8EsGGqLob4Enrox1pb/view?usp=sharing


︎Heatmap Testing

• Tool: Qualtrics Survey
• Testing Scale: 72/400
  
• Testing Scenarios:
  1. Add Items to Order 
  2. View Order Details
  3. Customer Support Accessibility
• Test buttons/hyperlinks click-through rate
  

- Scenario 1: Add Items to Order
“Imagine you recently received an in-Club pickup Order confirmation email from BJ's. You've decided to add more items to your order before it's ready for pickup. Below is a sample of the email we sent you. Please click the buttons or hyperlinks in the email where you would expect to add the item. Feel free to click on multiple options if you find more than one way.”

  1.1 Clicking Heatmap

  1.2 Browsing and Clicking Time Tracking
  1.3 Follow-up Questions: How easy or hard was locating the option to add items to your order? (72)


- Scenario 2: View Order Details
“Please click on the buttons or hyperlinks in the email where you would expect to provide more information about your order.Feel free to click on multiple options if you find more than one way.”

  2.1 Clicking Heatmap

  2.2 Follow-up Questions: How easy or hard was locating the option to view more order details? (72)


- Scenario 3: Customer Support Accessibility
“Imagine you recently purchased at BJ's.com and have questions or concerns about your order. You open the order confirmation email to address your concerns. Please click on the buttons or hyperlinks in the email where you would expect to find the answers to your questions.Feel free to click on multiple options if you find more than one way.”

  3.1 Clicking Heatmap

  3.2 Follow-up Questions: How easy or hard was locating the area that helped you resolve the order issue? (72)


- Wrap-up Questions: Please share any other thoughts or feedback you have regarding the email mockup.


*The complete User Testing Results Summary can be found here

High-fidelity Prototype

Pilot Release

In order to better verify the design updates and make last-minute iterations, we conducted a pilot release testing on 10% of the total orders. The pilot club release testing was scheduled for two weeks. After that, we would proceed with the full-chain release, assuming no issues arose.

Post-release:
Embed Survey

To better capture member feedback to help us iterate regarding the new order confirmation email template, we added a quick survey box in the email for real-time feedback from BJ's members.

︎Survey Results (09/27/23 - 10/24/23)

Post-Release:
Key Wins

︎Order Confirmation Call Volume
Since its launch in October, the Order Confirmation email improvements have effectively addressed member pain points, leading to a 27.7% year-over-year reduction in order confirmation calls as a percentage of the total post-purchase call volume.

︎Click-Through Rate(CTR)
Apart from a sharp decrease in Order Confirmation Call Volume, the post-changes Click-Through Rate(CTR) increased by 14 percentage points(pp), going from 1.7% to 16.1%.📈

Next Steps

Take Away

︎ Back to the top

Next ︎

---

Post Order Item Addition

Mosaic

Member Commnucation Optimization

Green To-Go

Cubitac

Janus

NYU Worklink

Access For All