Learning Systems Awards 2024

Niche.

One area I added is niche when identifying learning system vendors for the 2024 awards (end of the year).

It doesn’t mean that every vendor has to have this, but rather, it means exploring those who do and whether or not they find success.

I like to think of niche as a way to differentiate vendors because some accept all verticals and industries, and some do not.

Today, vendors are rumbling by the masses into the association market, which is generally recession-proof.

The problem, though, is that many vendors have no idea what area to focus on: trade associations, professional associations, or both.

Do they need to be able to integrate without an issue with an Association Management Platform (yes)?

Do they need to have already some processing providers they connect with, and will some associations need this because they have never done it? (Yes).

Are they familiar with ASAE? (They should become a member).

Does their system have capabilities specifically for the association space, which is relevant? (You better).

Do they know the #1 player and leader in the association market? I find many have no idea. And no, you have to find out yourself.

Sure, you will get some associations—everyone does. But if you want to be among the top five vendors, you must decide if you will be committed.

I always wondered why vendors are happy with being at the bottom or mid-tier in any market. It is as though they watch and understand the Premier League (English soccer), whereas fans get excited about finishing 8th.

I bring this up because it is a factor in my decision for 2024, not just with associations but across all market segments.

It plays a role, more so than in the past.

Niche and Christmas

I’m not sure how it works in your country, but many stores in the States put out Christmas stuff in July. I always wonder who sees this and says, “I must have it now, to prepare.”

You don’t see this with Valentine’s stuff—i.e., putting it out in November. Nor do you see it with Easter bunny stuff, nor Halloween—i.e., putting out stuff in April.

Christmas, though, is different.

I spent a whopping eight minutes, seeing if other places like the UK, with Harrods – the overpriced tourist trap, with stuff nobody needs – but hey it is from Harrods.

Oh, thank God. Harrods has over 780 products for Christmas in September.

Niche.

The stores that sell Christmas stuff are targeting customers who want it early assuming, no doubt, that they will get it for far less than if they wait until, say, November or December.

There isn’t any validation to this presumption.

Nevertheless, those stores see demand – and therefore aimed towards an audience – a niche.

While I am not eyeing a niche for Christmas here, I did place it as a small factor, as noted earlier.

The Approach for End of the Year Awards 2024

People always want to know my Approach, specifically what factors I consider when deciding which vendor ends up where.

My philosophy is simple – who is the buyer here? It’s not me – it is you.

What does the buyer see as relevant – from an overall perspective AND what the buyer should be looking at.

After all, the buyer has limited knowledge, generally speaking, of the trends to look at, what learning tech is in a system, why X or Y doesn’t make sense for adding in a system (not in 2024), and why you must look to the future.

Way too many vendors rely on what their clients tell them they want in a system. I always tell the vendor you are supposed to be the expert, not the client.

Why are you in this industry if you are not the expert? Have you ever thought about selling chestnuts around the wintertime?

Clients have purchased your system for various reasons – and wait for it; they are clients. Your job is to land new business. Did you read that? New Business.

A retort I often hear is that the potential buyer never mentioned X or Y, so it isn’t relevant or necessary at this time.

Again, who is the expert here? The vendor or the prospect. It should be the vendor.

I’m not saying that people lack knowledge here.

Rather, if you are buying a system, you better know this or this—or identify what are truly your needs—and not stuff I see like, “I need a system that offers a course catalog and can upload video.” Uh, every system can do that—but besides that, if you are relying solely on that as the key, then I promise you, you will end up with a system you hate.

The Factors

I decided to focus this year, on some different factors, new ones, plus some you have seen before. Each are weighted, and points assigned to each – with the maximum score being 150 points which is perfect. I am still looking at systems – so those vendors – surprise, I’m looking at the following:

UI/UX – This should be anyone’s number one priority. And yet, I’ve seen systems that look like they came from 2000 and yes, they land big clients. Personally, I believe they achieve this because they are really cheap OR they give the system to clients and such a low price point that the client can’t move on – it is just super cheap. I’ve seen this a lot. One company (name to be withheld) stayed with a vendor, because the price point on 22,000 learners was about $5,000 USD. And if the client went up with users, the pp would stay at 5,000 USD.

Anyway, I look at the learner side across the entire system, not just the home screen, because there are vendors who have a nice UI/UX for a few screens, and then the rest is awful.

Another aspect I focus on is the admin side of the system. The cleanliness of this interface is a significant factor in the overall usability of the system.

It’s important to consider whether non-technical users, without L&D or Training knowledge, can navigate the system without needing to contact support. A user-friendly system is key to its successful adoption.

If the system has been tossed over to HRIS or Marketing, for example, could that person figure out how to use at least the basics of it?

All relevant – and yet so many people ignore it.

The UX—user experience—is by far the most crucial aspect here, and it is looked at by those who have no idea what they are looking at, including folks I know from L&D who do not know what metrics are relevant.

All played a role here.

Oh, UI is essential – because people want fresh, and not yesteryear.

Metrics and Reporting – Relevant. Does the data tell me my learning story? It’s as simple as that. If you are providing customer training as your core – i.e. 60% or higher nowadays, ideally it should be 85% or higher (yes, the number has dropped due to many customers focused now seeking L&D, and on the other spectrum vendors who were strong in L&D, now going after or landing more customer training clients).

Anyway, do those in customer training have the right metrics that are relevant to people who overwhelmingly sell content or whatever and need to generate revenue?

Yes, I know plenty provide fee, still certain data is highly relevant.

View for example? Is not.

But the number of people selecting X content versus Y, which you created, tells me that something is wrong with Y.

Was it designed poorly?

Does it provide the right information? Is it static or engaging? If you are in training, for example, you need to figure it out—not the system.

And if you are seeking to generate profit – what content is selling the most? What is popular? What trends are you seeing?

If you have chapters – which ones are people going into the most – that tells me, that it is relevant to create another course on that chapter expanding it.

You want to see which courses/content are duds. This is essential, but sadly, it is rare for a vendor to offer this data point.

L&D has to include the skills tied to the content data point, such as how often someone is going into a course/content on that specific skill more than another.

What courses on soft skills are effective and which ones are not. I recommend you read my latest post on LinkedIn under my name – trust me you will find it very important.

Too many folks zero in on how long – that isn’t relevant – because they could have the course open to a specific area and be surfing the net on another tab or on their phone.

What is relevant is how often, what day of the week (you will see a trend here – over some time), trend lines that make sense and are relevant to your learning story, types of content to usage – you are using 95% videos and 5% authored courses. You see that your audience is going into the content that isn’t a video 82% of the time.

That’s relevant – However, you need to ask yourself why?

While many people do not believe in learning styles, I do—and have seen it firsthand, as have many people in training and L&D.

Unless you have been in the trenches, and in various business or markets, you wouldn’t know.

I couldn’t care what Fred on the net says from their research – You and I know it’s true.

Again, this data point is rare to find, but it is so relevant, and it is the preferred type of learning from a content side.

Anyway, I look at the metrics. And yes, the reporting, too. Can I see the report data visually, or do I have to download it to chop up stuff?

•Ability to assign CEU/CPD to content and capture data on the CEU/CPD by learner (seems so easy, but yowsa it is all over the map. It should be visible on the main screen when determining what course/content to take. Not on the page, where they must click to learn more about the course. Notifications and reminders are highly relevant here. Oh, metrics/reporting on this data plays a significant role.)
Notifications – Can you notify a bunch of people at the same time – tied to content – i.e., a group of learners, whether the content is ILT (boo) or online learning. How does the process work? Is it straightforward or convoluted? What is the process for notifications and reminders?
On the L&D side, the Manager dashboard, what can you do from the manager side? What can you see? Should you be able to see certain information? If it is debatable, what rules can the admin set up?
Rules are very important – and yet so many people ignore them.
Multi-Tenant – If you are an association or doing customer training/partner training, etc. OR you are in a company that owns their locations -let’s say X meal land, or different LOBs who want their child – think of you as the parent, and then the folks below as children. What can you do with it – uh, the tenant? Can you have a custom domain (where the vendor’s name is not in it)? What rules can you do – see relevance? Does MT cost?
E-commerce – if you need it, how does it look? Does a vendor charge you?
L&D here – HCM, HRIS integration – which ones does the vendor already have ready to go – i.e., API or a REST API? What systems can they integrate with today? If a prospect wants to talk to a client who has that specific HRIS/HCM system, can they do so? (they should).
Skills – Relevant nowadays. – Is reskilling BTW in the platform, and how does it work?
Workforce Development – total L&D side. If you are heavy in L&D and Workforce Development, tied around job roles and career pathways – how does that work? Do you have the right, i.e., minimums, to achieve success? I mean, goal management is relevant. A transcript is not.
Workflows (NEW)
Customer training features for the learner
L&D features for the learner
AI—It was weighted very low this year, but I am curious what they have today—ready to be seen with generative AI? And let me see it.
The big three are content creator, assessments, and skills pathways (growing). Okay, the big two are content creator and assessments. And yeah, that feedback loop is critical, but as noted, I can count on one hand how many vendors offer it. Oh, what LLM or LLMs are you using? Important, but nobody asks. Trust me; you will see the moment that it stinks or is poor with Y compared to another vendor. Lastly, does the vendor charge token fees? I don’t care how cheap they are; you either are charging a fee to the client or not. Pretty simple. For vendors who do not have AI, but it’s on the roadmap, I’m cool with that. And yes, people looking at a system are not asking about it as important. Nevertheless, uh, you are the expert, not them.
Support—I talk about this all the time. I want to stress this because a lot of folks never put it on their RFP, don’t ask about it in their demo, and ignore it—yet it is THE NUMBER ONE REASON people leave their system.

The vendors all have data on support ratios avg number of tickets per issue; they all have response time; they all have data on the top issues the learn about. Any vendor who says they do not – I strongly recommend not using them.

I hear vendors tell me they don’t know the prospect until they become clients.

I’ve also heard the NDA a prospect must sign to find out about support.

What are you – the secret police or a learning system? If a vendor makes you jump through hoops to learn about this data – move on. It yells out RED FLAG.

Ignore the NPS score—and if you are like, “I must know this,” ask them how many detractors they had (very relevant) and what their methodology was.

Funny, I find vendors never require folks to sign an NDA.

Demo score: I am looking at every system, going far deeper in the weeds than prospects go, and extracting all types of information. If a vendor shows me a deck (they always do), I ask them to send it to me after the call. Then I see how many do it. No surprise, I get vendors who never send it.

On the other hand, I will often ask a vendor to send me the deck ahead of hand and see if they will comply.

I can read and use my brain.

If I have questions, I will send them ahead of time.

Less than 40% of vendors ever do this.

I’m looking at that for my analysis.

Marketing – This goes to my marketing awards only – not for the above calculations – Does their marketing effectively work for their target audience? What is their marcomm approach? (Marketing Communications) This is basic stuff because I know a lot of vendors who hold dearly to this, as though they are in a lifeboat. None of the findings – i.e., what works or doesn’t- will be published for the record. Just #1, #2, #3
Best Vendor in Support – A new category. Every vendor says they have fantastic support. Trust me, I know who does and doesn’t.

That’s the list! I do look at some other lesser items, forward-thinking for one, but the above is the real stuff—the meat, if you will. Plant-based is recommended.

What Awards will be given out

Best Learning System (regardless of type)—The top five here. There will be two blogs, though, as in the past, with one covering 11-20 (not in-depth stuff, but enough) and then #10 to #1—this is the one folks want to know the most—and thus, I try to present the key data I found relevant.
Best LMS
Best LMS/LXP combo—this is common nowadays, but LMS is still the monster in terms of type.
Best Learning Platform – Any vendor who says they are none of the above – It’s like magic – okay.
#1 System for Enterprise (I will note #2 and #3)
#1 System for Customer Training (#2, and #3)
#1 System for the Association market (#2, and #3)
#1 System for SMB
#1 System for Support (#2, #3 noted)
Best Learning Technology for 2024 (think new products – I have seen one that is unbelievable, and others that are good – and yeah, go back to the drawing board)

As always, this is a global analysis – in other words, I am not focusing solely on the US. If you look at previous years, you will see systems from all over the world.

Bottom Line

As in year’s past, the rankings will slowly trickle out in early December. The Top 10 post comes out the second week of December.

E-Learning 24/7

Previous Award Winners (Top 10 only)

2023

2022

2021

2020

2019

2018

2017 (Part 1 #10 to #6)

2017 (Part 2, #5 to #1)