Solid and Safe Experimentation Environments with Ton Wesseling

[00:00:00] Sani: Welcome to NoHacksShow, a weekly podcast in which smart people talk to you about better online experiences. Usually I start the episode by saying you may know my guest from, not doing that today because you do know this person already, founder of Online Dialogue, organizer of the conference formerly known as Conversion Hotel, author of many online courses and workshops, Ton Veselin, welcome to NoHacksShow.

[00:00:20] Ton: Thank you very much for having me.

[00:00:23] Sani: It's a huge pleasure to talk to you. As I mentioned, as we were preparing, I took your course, uh, AB testing mastery, I think five years ago. And it was, it was my introduction to CRO and what an introduction it was. So, uh, first question I have for you really is you're from the Netherlands. A lot of my guests, a lot of optimization experts and really top level experts are from the Netherlands.

What's the connection there? Why are there so many of you from the Netherlands?

[00:00:50] Ton: My hypothesis is that it's a cultural thing. We have a history of conquering the world. We are a small country and as a small country, you need to trade with all sorts of countries and need to explore to be able to feature on country. So we always have this entrepreneurial mindset. I think that's needed, uh, to not be afraid to make mistakes.

And also from, um, hierarchy in organizations. Um, I think the best example I can give on how Dutch employees look at hierarchy and look at their boss is when I attended an event in Berlin in Germany, which is a different culture, the German culture. And I, as a speaker, had a badge that allowed me to go into any session.

In all the conferences that were, were out there. And one of my employees was a guest and only had a badge for the specific conference he signed up for. And I told him, well, at any point in time, you can just steal my badge and walk into a different session. So I was talking to this German guy who was the boss of an organization.

So he's formal. In Germany, you will listen to your manager and the manager will tell you what to do. This is, of course, the black and white stereotyping. Set up, there are good examples in Germany also. Well, this employee runs towards me, grabs my badge, takes off the badge, and runs off to the session he wants to attend.

And this German guy was like, Oh my God, you are, who is this? You're going to fire this man. No, this is one of my employees. And I'm happy with, we are on the same level. So in the Netherlands, you don't tell your employees what to do. You will give them room to explore themselves. So I think, I think it's a cultural thing.

It's the history and the way our country was raised and the way we, we look at each other helps a lot with the optimization mindset, not being afraid to make mistakes.

[00:02:43] Sani: That's a great explanation. Last week, I talked to Kevin Anderson, and he said something like, uh, we are not afraid to voice our concern and say what we think, basically, as a nation, and I think you need that. I mean, it's part of the growth mindset, basically. So, today, we are talking about something really important in this industry, optimization and experimentation, but any other, especially digital, where there's remote work, I think it's more effective.

So, we're talking about, uh, Safe and supportive cultures in, in experimentation. And you have been in this industry for almost 20 years now, closing in on two decades, was this kind of discussion possible 15 or 20 years ago?

[00:03:26] Ton: Now, of course, conversion optimization also started from an, I think, IT background, developer background, because you were able to create websites, you were able to develop something. Now, some people just wanted to experiment and test and measure. And the whole community was predominantly male orientated developers trying to hack the website to come up with better revenues.

Back in those days, there was not really a social focused internet. And people were afraid to share their ideas because, Oh, I have a little trick. That optimized my website a lot. So I'm not going to share this little trick, I'm going to keep this for myself. Um, and that changed a lot because of course over the years, at some point you start talking to people, the web became more social, Twitter started in like 2007, grew really big also in the analytics and optimization industry.

Um, uh, blocks were popular still back then, and people responded to blog postings. So people started talking to other people, uh, events started to pop up, and, and then at some point you will learn that if you share your IDs, that, that will take some, some effort, but you will get like eight IDs in return.

And that's a really good investment. And even if you share your IDs, you only get 0. 5 IDs in return. It's, it's still fine because it inspires you. So I think from there we learned. That's talking to other people is good. And then, of course, with the whole change of how Western culture is looking at diversity in the past 10 years.

Um, we really learned that different backgrounds, different perspectives in a simple way, it's male, female, but different ethical backgrounds really helps to have a different look on how the consumer is using your products. Because in the end, the consumers are also diverse. They are not, not only like Old, white, male consumers you're optimizing the website for.

[00:05:27] Sani: Thank God it's not like that. Uh, so you, you mentioned this is a very diverse industry. And, and every week when I talked to a guest on this podcast, I asked them about what their origin story with optimization is, and it's always different. So my question to you is how did you get into optimization?

[00:05:44] Ton: Uh, I fell in love with building websites back at university in like 1994 or something, 95. You had the first internet browsers. I was able to create my own little HTML website without having to use anybody. I could do this myself. I could publish something that people could use. I could produce something.

That people could use myself. And that was wonderful. It's like, it's like playing with Lego and you build something amazing, but no one can see it. No one can use it. And suddenly you could build something and people could use it. And then of course, I had my own web server that has log files. And I was looking at these log files.

See, oh, I have 24 hits on my website today. Oh, what can I do to make it 35 hits on my website? So this is, this is why. My love for data and, and, and optimization started to run by just looking at log files and I have a passion for numbers. So that helps a lot. Um, uh, so it's more analytics at first, but then when I really learned how you are able to, to run a B experiments and back then it was not really a B experiments.

I just made a change and then track the change over time to compare it to the different variation before the change. Um, uh, but that was really rewarding. It's like, The, the, the, the energy you will get from trying to solve this murder mystery and being the Sherlock Holmes, trying to solve this puzzles, build a bigger, better Lego castle, that's really fun and rewarding and that brought so much energy.

So then I really fell in love with conversion optimization, which was 2003, I think, by Brian and Jeffrey Eisenberg and just kept sticking to this industry since.

[00:07:25] Sani: I love how you called it the murder investigation and trying to understand and starting with the log files. That's really an investigation. So that must have been interesting and fascinating. Um, working by yourself, no one understands what you're doing 20 years ago, 25 years ago, no one would understand why is he reading the log files, but it must be, like you said, even more rewarding now in the social internet where you can share this, you can, you can share eight ideas, get 20 back, or even to get 1.

5 back, it's better than zero and keeping everything for yourself. So I love the way the industry is progressing. I think it can only get better from here basically, especially now that people are focused on diversity. And. Making sure the industry is diverse and not just old white males, like, like the customers you

[00:08:13] Ton: Yeah, it's not only about diversity. It's also about creating a safe environment where you feel at ease to express your ideas. Um, uh, and of course, diversity helps a lot. Otherwise you will get locked into, uh, only other white males will be happy to share something. Um, but the safe space is so important, of course, for every industry, but especially for experimentation optimization.

Because you will make mistakes. You need to try and fail to learn and at some point come up with a better idea. And if you don't feel safe at failing, then the team you're working in will not be a really successful team. It can be optimized. And that safe space and diversity helps a lot, is really important in experimentation.

[00:09:00] Sani: Right. And at a personal level, adopting the growth mindset, it's not easy, but it's easier at an organizational level. Do you think there are a lot more challenges? Like why doesn't it happen with every single organization? Because it seems like it's the logical thing to do. Right.

[00:09:16] Ton: Yeah, it, um, it takes time. It takes time. And, uh, especially Managers, um, letting go of decision making, um, that's like the only power you'll have. Like, like the example I gave you from the BATS in, in Germany. Most German, German companies I know of, uh, will, will have a team waiting for the manager to tell them what to do.

And if you take away that power from the manager also, the team is going like, Oh my God, now we have to come up with something. But then also the manager only has one power. Make decisions and then suddenly decision making is democratized by experimentation and validation, which is kind of scary. So their role should change to become an enabler for that team to shine and then take proud of the fact that you were the coach that was able to create a successful team.

But if you were not raised that way and have been working for 40 years. Being proud of the ones, the one that are, is able to make decisions in an organization. Uh, yeah, that's, that's, it takes time. It really takes time to take those steps. And it differs per culture and maybe it's more easy in the Netherlands because of our specific background.

Just a hypothesis. I don't have the data to back this up. This is just my experience and it's one. But yeah, it takes time and the bigger the organization, of course, in smaller organizations, leading by example, if you have a founder that has an experimentation mindset that has a growth mindset, it's so much easier.

[00:10:46] Sani: Uh, and what you just said, this is why the best athletes are rarely the best coaches because the mindset is completely different. I did this on my own and Andre Morris talked about this on the podcast as well. The traditional mindsets in, in, in getting promoted. I was right. I was promoted. So if I'm promoted this high, I'm right all the time.

We don't need experimentation. Just listen to me. It's a complete organizational shift to get to that growth mindset from, from the bottom up, I guess. So. We talked about diversity and also collaboration with the social internet as the key to innovation. So what, what kind of innovation has that brought us in the optimization sphere?

Hmm,

[00:11:31] Ton: awareness of consumer psychology. Because Our background mostly coming from development and maybe some some user experience that also already touches a little bit to consumer psychology. Um, well, we come from a data driven background. We just look at numbers. It's the way I started my agency online dialogue.

I was just an analyst doing A B testing and the tests I created were like changing button copy, bigger buttons, USPs added to it, switch the image, uh, all sorts of usability tricks. To optimize conversions. And then I met my business partner, Bart, who is a consumer psychologist. And he was working at a web agency where they created new websites.

So responsible for design. And based on his theoretical thoughts on consumer psychology, they created new websites. But they never tested this or validated this. So he was like, oh my god, you can test this? And I was like, oh my god, we should use your consumer psychology knowledge to run experiments.

[00:12:31] Sani: at first sight, basically, right?

[00:12:36] Ton: Uh, that's what we do. Add motivation, take away friction. It's all consumer psychology. So, um, I think the openness, the social web opens our eyes for different industries. Also the shift from conversion optimization being part of marketing to becoming more part of products and really optimize the whole digital products of a company.

That's all because we started talking to each other and start seeing, oh my god, this UX department, this UX people, they do the same thing as we CRO people. Maybe we should talk to each other.

[00:13:12] Sani: Right. But that's called, uh, industry maturity basically, but CRO 30 years ago, no one considered it, I mean, 25, 30 years ago, no one considered most people didn't consider websites 30 years ago, but 25 years ago, CRO was not a thing, 20 years ago, barely a thing without a name almost. And now it's, it's in everything.

Uh, yeah, but, uh, to go back to, you know, feeling safe and valued, uh, as an employee. How much of a problem is that really from, from your experience in, in, in optimization industry, that there are people who don't feel like they're safe or valued?

[00:13:50] Ton: If I look at the success of teams, some companies are large enough to have several teams, and if some teams don't feel safe 100%, less people will speak up.

[00:14:03] Sani: Hmm.

[00:14:04] Ton: And what I measure in these teams that they are less successful. So the teams that are operating in a safe space where everyone speaks up, where the teams are diverse from different backgrounds, from different industries, different, different ethical backgrounds, different genders, they are more successful.

And I think this is a combination of, um, uh, team happiness. Because if you're safe. Uh, there will, you will be less afraid, you'll be more happy, and you also need that happy energy to keep on going because in the, in the end, you're just repeating and repeating and repeating in your work. And if you're working in a non, non, not so fun, not so safe team, you will have less energy.

But also the quality of the discussions they're having, looking from different perspectives, different backgrounds, really helps to come up with better ideas. So in the end, they're more successful. And sadly, my data is like too low to make it like a significant result. I've read on some papers that did research about this, mostly published by big consultancy firms, like Accenture or Deloitte.

Um, so of course they also have a goal with publishing this kind of papers, probably a commercial goal. So I don't trust that kind of research too much. There must be more research. Hopefully companies like, uh, Microsoft or Amazon or, or booking in the Netherlands will really start investigating in the teams.

Why some teams are more successful than other teams. And could this be because of the diversity? Of the teams, yes or no, that would be really interesting. But so far, the data I have, low data, like 30 teams, I do see a difference.

[00:15:46] Sani: But you can also draw a parallel to, uh, an autocratic regime or country. Like those countries in the long term usually don't do Versus something where a person, an individual is free to express their thoughts and it kind of leads to a more prosperous society. I mean, not in my opinion, that's just a fact.

[00:16:09] Ton: I remember an A B experiment from an international company that we worked with that had a checkout page in the Netherlands, and also a checkout page in an e commerce company in Poland. And in Poland, the variation where a senior person, almost like a military person, Told them about the next steps that were coming up and what they should do with those next steps.

Really optimized conversions. And beginning before it was, okay, this person really explains what's going on. So feeding forward, that probably helps. When we tried this in the Netherlands, it really tanked conversions. They went really down because nobody likes someone to tell them what to do. But in Poland it really helped.

And then we suddenly realized, oh, it could be that this is because of their background. They are used to the fact that someone is telling them. What to do. So then we started to elaborate, uh, to continue on that and iterate and come up with more experiments in that specific direction and really learned that it doesn't make sense to have the same checkout flow in Poland as in the Netherlands.

As it is probably in Serbia or Portugal or Sweden and so on.

[00:17:16] Sani: Every country is different, exactly. And yeah, me growing up in Eastern Europe, in Yugoslavia, I get what you're saying. What you did in Poland, I get why it works because it's an authority figure, just like, uh, uh, white lab coat is for some, I mean, that, that's like the hypothesis is that it works usually does, but that is a very interesting thing and something that I think most people would not even consider that we need different, uh, authority figures for different countries like that in Europe, countries that are a few hours away from each other,

[00:17:47] Ton: Yeah.

[00:17:48] Sani: that's,

[00:17:49] Ton: Yeah, and the most interesting thing is that I know a lot of agencies have learned this and companies have learned this. in optimizing for their customer, for their user. But they're still not applying this in the optimization of their own teams.

[00:18:04] Sani: Oh,

[00:18:06] Ton: the same mindset. If you know that some teams need different steering in some countries as your website should be different, then you should also apply this to become more successful.

Optimization works both ways. You need a successful team and you need a successful hypothesis to optimize for your users.

[00:18:24] Sani: And do you see that happening anytime soon for most organizations or is just something that they will never think about?

[00:18:31] Ton: the only thing I'm afraid of is that a lot of visible companies like on the stock markets, large brands are only playing the diversity card because they have to, because their shareholders believe they have to, um, and it's, it's a start, uh, but, but hopefully it's not just because this is something they have to do.

Hopefully it's, it's something that they really believe in and put their heart in and not only their, their heads.

[00:19:01] Sani: Like you said, it's probably just a start and hopefully a step in the right direction. But, uh, I do get that feeling as well, especially for, uh, The largest companies that you can imagine, like when they play the diversity card, they really are just playing the diversity card is what it seems like. It's not really fighting to change something.

It's just our marketing department told us we have to, that is what it looks like. Sometimes I'm not trying to get canceled or anything like that. I think it's a step in the right direction still.

[00:19:29] Ton: And still, we're still part of a society. If you look at LinkedIn, if you open up your LinkedIn feeds and look for the hashtag experimentation or whatsoever, you will only see positive stories. You will only see examples of stuff that went really, really well. And only a really small percentage of people is sharing stuff that went wrong.

So we're still in that culture where it's the norm is to share wins and don't speak about something that's not good

[00:19:59] Sani: That's a

[00:20:00] Ton: or seen as not good, because it is good, but seen as not good.

[00:20:03] Sani: That's a great point. And one last question about this safety and, and, and, and culture in experimentation. How will AI affect this aspect of working in experimentation? Not from a technical standpoint, technical solutions and everything, but from a human and being connected kind of standpoint.

[00:20:21] Ton: So far, AI for me has been just another team member, um, that, that you can have a conversation with. Uh, that, that helps in, in, in hypothesizing, in brainstorming, coming up with ideas. Um, so it's a fascinating new partner. Um, I'm not sure if AI itself is diverse. Uh, if I look at chat GPT, uh, to me, it seems like, uh, more American, like persuasive marketing students.

Um, uh, but of course you can tell, please be a, uh, be, be, act like someone else. Um, uh, but it, it, it speeds up a lot. It speeds up research a lot and so on. Uh, if it, if it helps with diversity, Hmm, not sure yet. Not sure yet. I don't

[00:21:16] Sani: So the, the jury is still out, basically, I have a question from, from a listener and a personal friend, Luka Nikolic, you may know him from LinkedIn. He's a, of course you do at what point, before we go to the, get to the conference doc at what point in terms of number of tests being run, should a CRO team start measuring program metrics rather than looking at individual test numbers?

[00:21:39] Ton: Uh, I would say above 100.

[00:21:44] Sani: Okay.

[00:21:44] Ton: you pass that mark, it makes sense to store stuff in a database, uh, to come up with learnings, meta analysis, and start looking at, at, at program metrics more than, than just building, testing in an

[00:21:58] Sani: Individual tests. That's a great question. I'm sure Luca will be happy to hear this. The conference formerly known as conversion hotel. I've heard other names. I've heard people call it, uh, experimentation island also, uh, unofficially. So, uh, I love a good origin story. Tell me the origin story of the conference formerly known as the conversion hotel.

[00:22:20] Ton: Um, I was hired by commercial conference organizers as an industry expert, experts to help them, uh, assign speakers for conversion conferences, analytics conferences and so on. And these conferences were always one or two day conferences. Uh, back to back talks, not a lot of breaks, a lot of sponsors. And once the name was built for the conference, they started on minimizing on costs, like cheaper location, uh, more expensive tickets, uh, less money for the speaker lineup, uh, uh, cheaper sandwiches, uh, just to make money, make it to cash cow.

So I got fed up with the business model of building a brand and then demolishing it. In the next five years to make money, um, and I begged like, please, we need more breaks. We need more social time. We need more interaction because that's the value of meeting people. This was even before the pandemic, like we need to meet people and talk to people and not just sit there and listen to just a couple of speakers.

Uh, they did not listen because they thought it was like a too big risk to have like a multi day event, including hotel stay. So at some point in 2014 I said, I'm going to organize this myself. And, um, the, uh, my majority of attendees in my first year will be from the Netherlands. And one of the habits of Dutch people is they want to sleep in their own bed.

They want to go home. So a multi day event, they want to go home. So can we find a location where you're just not able to go home? So we ended up in an island and we, the last session will end, uh, will end after the final ferry leaves to mainland. Uh, hotel stay is included and thus people are also like, Oh my God, I paid for this hotel.

So maybe I should use it. Um, uh, and that, that became Conversion Hotel known as Experimentation Island, uh, in 2014. And then we started growing, growing, growing. And in the, in the first five years, uh, I've learned. Looking backwards that we focus too much on quantity. We wanted to have more attendees because with more attendees, we will have more money.

We can hire better speakers, have better sandwiches, better dinners, better parties, better live bands and so on. And then after year number five, when we were almost 400 people. Oh my God, this is too big. It ruins the quality of the conversation. So then we redesigned it. We redesigned to the conference formerly known as Conversion Hotel.

Again, did this for five years in a row with focus on quality, less keynote speakers, more social time, more on conference sessions, more roundtable discussions, a maximum of 250 attendees, but still trapped on the island in November. It's shitty weather. It's raining. It's cold. You want to stay inside in the hotel, which is good because then you start talking to people.

So, and that's what I've been doing for the, for the past five years. So our 10th edition was last November. And again, uh, uh, I'm going to redesign, uh, this whole thing because, uh, it was so successful. Our net promoter score was like almost 100. You cannot go higher. Um, so I'm afraid that if we just continue in the same format, um, the expectations will become too high. So I'm going to tear down the puzzle and look at the beautiful parts and try to create a new puzzle with lower expectations and then start something new again. And that then maybe becomes like the conference formerly known as the conference formerly known as conversion hotel. Maybe that's not the best name, maybe we should call it Experimentation Island.

Maybe it will be the Unknown Conference, I don't know. But there will be something happening again, because I know a lot of people are afraid, Oh my God, this conference stopped. And something, it's not going to happen again. Something will happen, but what it will be, I don't know.

[00:26:05] Sani: that's a beautiful cliffhanger, uh, to, to almost end the episode, but is it, has it been the same island for 10 years? It's what I want to

[00:26:12] Ton: Yes.

[00:26:13] Sani: has been. And

[00:26:14] Ton: Same island, same location.

[00:26:16] Sani: I love that story. I love how you started that. And I love the fact that we're not going to say anything now, but there might be some news soon.

Like you said, preparing for the recording. Don, this was an absolute pleasure, to finally get to talk to you after. going through your courses, following you on LinkedIn and not attending the conference last year. But I saw the wrap up video, which is just makes me feel even worse for not being there, but hopefully this year.

And yeah, I really look forward to meeting you in person sometime later this year and to everyone who was listening to this episode, please consider sharing, reading and reviewing the show and I'll talk to you next week.