Results of my startup customer research survey for the Brewery SOP tool

Recently I found myself with the opportunity to work full time on something new and I began exploring some business needs around Standard Operating Procedures (SOPs). Having started and managed a brewery for 5 years, I know full well the challenges in having a workforce trained up and compliant while dealing with all of the other day to day issues.

The first thing I wanted to do was get some intel from other people to ascertain whether others have this problem and how people are approaching it, so I decided to undertake a customer research survey to begin to test some of my own hunches and assumptions. You can read about the process and my thinking in the post below:

Related posts:

• My Startup Customer Research Survey
• Is Startup Validation Bullshit (2013)

In this post I will run through the actual results of the survey. This may be interesting to you if you are a founder wanting to follow along with the process, or a brewery owner interested in what other breweries are doing.

Brewery-specific report (free download)

I have put together a PDF report of the results specifically for breweries. The 12 page report provides the results including averages and comments around SOP usage, reasons for using SOPs, software used, challenges and much more. Click here to download the PDF written report, or continue reading on for the overall results and some lessons and commentary from me on the process.

Download report

Or continue to read on and in this post I’ll run through the high level results and my own learnings and commentary.

Completion goal

I set myself a goal of getting 60 completions. I thought this would be tricky because I don’t have easy access to an audience of brewery owners, however I did also encourage non brewery owners to complete the survey just in case I couldn’t get the numbers.

In the end I was stoked to get 153 completions of the whole survey, 99 of which were brewery owners (65%). I should be able to get some very good info with that many responses.

The results I present in this article are for the whole audience. However at the end I’ve got a section to wrap up the brewery-specific results vs the non breweries, and there are some surprising and large differences in some areas.

Overall results against assumptions and hunches

I came up with a list of assumptions and hunches before I did the survey and then I compared these to the actual results. Here are the results in a table showing whether I nailed it or not.

Assumptions

= most likely invalidated

= unsure at this stage

= most likely validated

This is my list of things that need to be true to be able to pull this off.

Y/N	Assumption	Result	Comment
	SOPs are important to a decent amount of breweries.	85% say yes	For this one I asked if people currently have them at work. I guessed this number would be around 30% but for breweries specifically it ended up being a whopping 85%. I also asked how often they used the SOP tool and the most popular response was every day. I definitely feel like it was a very engaged response which tells me that SOPs are something that brewery owners hold in high importance.
	SOPs are a big enough problem for brewers that they will pay for a solution.	15% currently paying	I asked if people’s current solution was free or paid and 85% were using free solutions although 15% being paid is not an awful result. I also asked if they would pay and 37% said yes which was much higher than I expected. This definitely does not ‘validate’ the idea, survey data is notoriously unreliable especially on questions like this with so many potential variables and so many things that could potentially get in the way. Still I’m encouraged by the results and I do believe there are a decent number of breweries out there willing to pay for a solution for this.
	The current tools on the market aren’t adequately solving the problems for breweries.	Lots of issues with current tools	Overall I would say generally people aren’t super stoked with their current solutions. Lots of people were looking for SOP-specific features like reporting, and most people were using word processing apps which are never really going to give you everything you need for something like this.
	Using phones for SOP completion will massively increase completion rates	Unclear	This one I can’t test with a survey, I would have to build the app and see how it goes in a real setting. There definitely didn’t seem to be a huge desire to use phones for this though, most people are using computers. But I do think that a better mobile app and potentially using AI, could sway people in this direction. The jury is out on this one.
	I am able to get the attention of enough breweries to get paid customers without spending money.	99 completions.	I would consider this one pretty much validated. I had a large number of completions, between Facebook groups, podcast and organic social media and knowledge of paid channels and my connections in the industry I think I can access the initial audience. I had 99 brewery workers complete it, 97% of them showed some interest in checking out what I put together, so I am very encouraged by this.

I would certainly not consider the product as a whole to be ‘validated’ at this stage, but I am quite encouraged by the results.

Hunches

This is a list of things that I suspect are true and I thought it would be useful to use the survey to test to see how in touch I was with what’s going on.

Y/N	Hunch	Result	Comment
	SOPs won’t be super important to smaller customers like brewpubs but will be seen as important to production breweries	Don’t think this is true	I’m not so sure this was true, most breweries saw the importance but smaller breweries were less likely to want to pay for it.
	People care about SOPs for 2 main reasons, Safety / Compliance and Consistency. If this is true, a solution that increases completion will be well received.	Confirmed, consistency and compliance were the top reasons.	Not too many surprises here.
	People might have played a bit with note taking apps for SOPs but my hunch is generally people don’t use phones for them because there are no good mobile-first solutions.	Confirmed, most people are not using phones (85%).	There was however less use of note taking apps than I thought there would be.
	I suspect there is a bit of tension in creating and completing SOPs because it’s seen as something pushed down by managers because they are created on computers by people who have computers at work. One major benefit of doing it as a phone solution is it would make it a lot easier to change this model. All frontline staff have phones, they could create the SOPs and managers could edit / sign off on them.	It’s definitely true that they are mainly created on computers although staff engagement with SOPs seemed to be reasonably high.	I didn’t specifically ask people if they were a team member or a supervisor which perhaps would have been a good idea.

Overall I’d say I was reasonably on the money with those assumptions and hunches and the only major surprise was there seemed to be a pretty high desire to have a look at what I built if I build a mobile product here – and a high desire to pay for it if it meets the needs. Again however, I’m not reading too much into that at this stage.

All questions

Here are all the averages for the questions I asked and a few comments from participants and comments from me.

Do you work for a brewery?

53% Yes: 99 out of 188.

Comment: I had a goal of getting 60 responses from breweries so this is quite good. I would have preferred the completion rate for breweries to be higher than just over half, but I also didn’t have a list of brewers to send it to and was relying on a few Facebook groups and my own social media profiles and personal email list so it’s not surprising there are a lot of people from outside the beer world completing the survey.

Also just so you aren’t confused, 188 people completed this question but only 153 completed the entire survey because of people dropping off.

I’m considering creating a tool to enable supervisors and staff to easily build and roll out Standard Operating Procedures (SOPs) on their phones. Think things like processes for dry hopping, forklift pre-start checklists, completing a packaging run, or maybe even outside of production like closing the Taproom or completing a sales call. Do you currently have any SOPs like this at work?

74% Yes: (128 votes)

Comment: This one blew me away, I thought it would be mostly smaller breweries filling in the survey so I thought this number would be well under this, maybe somewhere around 30%. I think this is a good sign for breweries and also a good sign that there potentially could be an opening here to help them with SOPs if I can find the right kind of solution.

For people who didn’t have SOPs at work I also asked why. For most people it wasn’t one of the prescribed options but it was generally some variation of being in a small team and not having enough time. Here are a few typical responses:

• “I have them, but at the moment it’s only me out the back. To do them takes time away from production work, and only to put in place for myself.”
• They are in planning stages but never seem to get done as we are all too busy

Phew that would be awkward if you didn’t! Can I ask you, what would you say are the main reasons you have SOPs at work?

These results were as I expected. The main reason is keeping things consistent (29%) with safety / compliance being a close second (24%) and reducing knowledge drain when staff turnover coming in third (19%).

There were some great comment responses to this question as well:

• “I believe in documentation or it didn’t happen. So we make sure SOPs follow the beer. If something goes wrong with the beer later, I can look back to see who did what, when, and were there any documented deviations”
• “Creating a Standard/Benchmark for all new staff to aspire to.”
• “For new hires to be able to review when imminently completing the task, are a little unsure of next step, they can get their answer instead of having to call someone who is off shift”

And what would you say were the main challenges of creating, maintaining and rolling out SOPs?

This one had quite a few close responses including Taking too long to create (22%), getting people to create them (20%), content going out of date (18%) and Getting staff to complete them (17%).

Getting brewers to use computers was not high on the list (5%) which either means they have no issues getting brewers to use computers, or there are already good options out there using phones. This isn’t the best result for my product idea but let’s see what the rest of the data says.

I’d love to know what tools are currently being used in your organisation for managing SOPs.

Some interesting results in this one too with word processors being the tool of choice for building SOPs (Google Docs 27%, Office 24% and ‘Other’ being 3rd.

In the ‘Other’ category there were loads of options including Notion, Asana (came up a few times), SmartSheets, SkyTrust, Qudos, Articulate, Sweet Process, Confluence, WordPress and Basecamp.

It’s interesting that the vast majority of these tools are not dedicated SOP tools. That tells me either it’s not a category that needs its own tool or there could possibly be an opportunity there to do some specific things that wordpressing and note taking apps don t’ generally do

Also I had to include this comment because it cracked me up and I can relate “A, B, C, D, H. It’s an absolute clusterfuck.“

How often do people at your work use an SOP tool?

This one surprised me and I found it encouraging. 26% chose multiple times a day and daily was the next most popular option (25%). Also a few comments in this one indicating an issue I expected to see being that managers want SOPs to be done but they find it hard to get staff to do them.

• “I wish they would”
• “They don’t use them. They just documented the processes.”

Who normally creates the SOPs? Team members, Supervisors or a bit of both

“A bit of both” was the most popular response here at 56% which I supposed is encouraging, second was managers at 37% with only 6% for staff. That said its probably not that surprising that not many people only have staff completing them because generally you’d need a supervisor to edit / review them.

The other options that came up here was a third party like an external consultant or training coordinator. And there was a fair bit of sentiment around what should ideally happen with staff creating them and supervisors reviewing them:

• Managers have to approve, but frontline staff are the ones who write it.
• Ideally, front line staff should create the sop and managers need to approve them.
• However, I think staff should be part of the process.

Have people at your work ever created SOPS on their phones?

This one was a very overwhelming % of people saying no at 86%. This was definitely higher than I thought, I thought at least people would be using note taking apps or a few of the available safety / checklist apps to at least start the process but it seems very much something that happens on computers.

The question that remains unanswered is why. There were some comments around SOPs being too wordy to complete on phones and the tools of choice probably tell the story (Office, Google Docs) that a lot of these are too wordy to write on a phone.

That said I’m still not convinced you can’t create the majority on the phone or at least complete them on your phone – especially if a few things happen:

• Phone use would encourage more images and less writing which I think would ultimately be a good thing for a lot of SOPs.
• AI can assist in writing some of the words for SOPs. If that’s true then it would require a lot less words and would make more and more sense to do it on the phone.
• I’m not aware of any really good mobile SOP phone apps so perhaps if there were some, they would be more widely used.

This one wasn’t the most encouraging result for what I’m doing. Generally I think if you want to look at what someone’s behaviour will be you look at what they are currently doing already and if they aren’t using phones it’s not a great sign. But I’m not quite willing to give up just yet because of the 3 reasons above.

Is there anything you don’t like about your current solution?

The most popular response for things people don’t like was ‘not great at reporting who’s completed the SOP’ at 29%. This one I find very encouraging. We know the most popular tools for creating these are word processing apps but those apps don’t tell you who’s completed the SOP.

That means if you want to succeed in the 2 main benefits of having SOPs (consistency and compliance), the current solutions aren’t meeting the need.

You would have to think a dedicated SOP tool that could create content as easily as Google Docs but also reported on the results would be a popular tool.

I’m also very encouraged by some of the other comments provided in this section. There seems to be a lot of people who aren’t too happy with their current solutions:

• It’s cumbersome and slow mostly due to admin. Utilizing AI and or even Fiverr help to translate notes into documents could help.
• Very slow and clunky to operate. Slow and clunky to build. Just downright miserable in nature
• I would like it more collaborative, and also stored in an easy, central location that everyone knows and has access to.
• We use word docs converted into PDF’s. They are kept in shared drives. Clunky and slow to access and no interactivity with SOP.

Is your current solution free or paid?

For this one ‘Free’ came in at 73% by far the majority of people which wasn’t really surprising to me. However I suspect this has a bit to do with the team sizes. For larger teams most of these tools are paid, even Google Docs generally would be used via G Suite for large teams and would be a paid product.

Comment:

If I build a sweet mobile app that enables you to easily create SOPs and have team members complete them would you use it?

This one I hesitated to ask because I wouldn’t really trust the responses for a question like this. I expected the results to be under 5% for this and even if they were 5% I didn’t expect those people would all translate to paid customers (that’s not really how this works).

That said, I was very very surprised with the response to this one with ‘Yes and I would pay’ being chosen by 36% of people and ‘Yes if it was free’ chosen by 50%. I would say these are very encouraging results, however I would also say that when it comes time to actually using something and paying for it, there is a hell of lot of things that get in the way before you translate these results into actual paying customers.

It surprised me particularly since the rest of the survey didn’t really indicate to me that there was a huge desire for people to do this on mobile.

A few comments:

“Don’t be a dick and charge like a wounded bull. The industry needs something like this, but it needs to be affordable.” – This one made me laugh and is also true, this industry isn’t one that has a lot of spare cash to throw around.

“I’d certainly review it but unless it’s properly controlled in terms of historical change history and approved changes it will not be usable.” This kind of feature came up a little bit so something to think about for sure.

Breweries vs others

Since the above data includes all participants, I’ve put a little table below with the results for non breweries vs breweries and some commentary on the results.

Question	Brewery	Non Brewery	Comment
Do you currently have SOPs?	85%	60%	Very interesting – seems to be important specifically to breweries.
Main benefit	Consistency (30%)	Consistency (27%)
Main challenge	Takes too long (24%)	Getting people to create them (22%)	I suppose indicating that the brewery audience is particularly time poor.
Most popular tool	Google Docs (28%)	Google Docs (26%)
How often	Daily (22%)	Multiple times a day (34%)	Interesting – could possibly be a workforce that is on computers more often.
Who creates them?	A bit of both (52%)	A bit of both (64%)	Interestingly ‘Frontline staff’ was 9% for breweries and only 2% for non breweries indicating that breweries do seem to be a bit more on the front foot with staff completing SOPs.
Phone usage?	No (86%)	No (86%)
Things you don’t like	Reporting (28%)	Reporting (32%)	But in both cases ‘Other’ was the most popular.
Free or paid	Free (85%)	Free (54%)	Very interesting and perhaps not super encouraging. Breweries seem less likely to want to pay for a solution to this.
Would you use and pay for an app if I made one?	Yes (37%)	Yes (35%)	More interesting though is out of the non brewery audience 32% said they were not interested but in the brewery audience only 2% said they weren’t interested. I think it’s pretty clear that breweries are specifically interested in this, but they are probably a hard group to get to pay for something like this.

Conclusion

Overall I was pretty intrigued and happy with the results of the survey. if you consider how often ‘time poor’ came up in the comments, I’m very grateful that 153 people decided to complete the whole thing. I also got some responses from some very well known breweries around Australia and overseas. So if you were one of them then thanks for doing it, I very much appreciate it!

Where to from here?

Based on these results I’m definitely still keen to pursue this idea, I feel like there’s something here. Although it won’t be easy to arrive on a set of features that a lot of people are prepared to pay for. The next step is to build an MVP (Minimum Viable Product) and get a few people using it for feedback, let’s go!

If you are interested in being updated on when I bring it out, or you just want to follow along on my journey jump on my weekly emails here.

Cheers.