Cammy Bean’s Learning Visions: DevLearn 2023

0
Cammy Bean’s Learning Visions: DevLearn 2023


DevLearn has always been my favorite L&D, elearning-nerd out conference. It’s where I first hit the stage back in 2009 with my Kineo colleague with Stephen Walsh and then most every year until about 2019. And then we had a pandemic and I sat out a few years. 2023 was my first DevLearn since 2019 and it was really great to be back!

Photo of four smiling individuals in front of a conference registration desk with the Kineo logo behind them.
  • Kineo was a proud sponsor  of DevLearn this year and it was super cool seeing our logo at the registration stage and on the big screen.
  • Amazing to be on the ground with three of my Kineo colleagues: Joe Replin (Technical Consultant coming to Vegas from Chicagoland), Rodrigo Bolanos (Head of Kineo LATAM from Argentina), and Vicky Bartolacci (Kineo Managing Director all the way from Sydney Australia). 
  • So great to connect with so many amazing people – old friends and new connections made. 
  • Data and Analytics are top of mind for people and extremely low on the L&D team’s list of skills. Orgs and individuals are eager to build this capability.
  • AI was the darling – and it’s coming soon to a workflow near you if it hasn’t already. AI is transforming how we work and what we’ll be delivering to learners. Game changer. People are feeling a lot of uncertainty and fear and…yes, excitement. Hold on to your hats, the ride is just getting started.
  • Ooh – and I signed a whole bunch of copies of The Accidental Instructional Designer: Learning Design for the Digital Age. Always fun to connect with readers and fellow accidents.
  • And finally, Vegas has amazing food – no doubt. AND I got to experience the Sphere (we didn’t splurge for U2 tickets, but we did see an amazing $90 movie! Hold on to your seats, it was awesome.)

Here’s a run down of the sessions I attended (and some I led!).

Morning Buzz – AI and Your L&D Role

Josh Cavalier

Josh is a big AI player right now in the industry,
definitely one to follow.

Don’t wait. Start now. Things are changing FAST. We don’t
know where things are going to be in 3 months, let alone a year. Everything is
changing at rapid pace, but we do know that AI is here and it’s disrupting
everything.

Lots of fear in the crowd – “what’s this going to do to my
job?”

Get a corporate policy in place.

Policy > Manifesto > Playbook

Manifesto – how your L&D team is going to use AI

Playbook – what tools you’re going to use and how – and
then meet on that playbook monthly (this idea actually came from Marcus
Bernhardt who was in the crowd – also an AI guy)

Opening
Keynote: Phil Rosenthal

 

Phil
Rosenthal was a producer/writer of the show “Everybody Loves Raymond”
and now has a show on Netflix – 7 seasons – “Somebody Feed Raymond”
where he travels the world and eats good food and makes people laugh.

 

Use your
constraints – working within your box can unleash your creativity.

 

The show
Raymond – they wrote it to be intentionally timeless. Keep out any time bound
references (e.g., you don’t mention Bill Clinton in the 90’s) and you make the
content RELATABLE.

 

The best
stories are those that are relatable. As a storyteller, satisfy your own
interests and be specific. When we are specific, people are more able to relate
to your story.

 

“Do the
show you want to do because we all get canclled.”

 

Creating a Robust
Company Learning Culture with Values-Driven Development

Jennifer Gilley,
Anuvu; and Amy Wheldon, Go1

Building a
values-driven learning culture at an org obviously needs to start with stated
values. Then follow a methodology:

  1. Build an authentic FOUNDATION
    that centers on philosophy, values, and the employee experience.
  2. Execute meaningful form – uncover
    the opportunity, informed strategy, culture shift, feedback loops
  3. Leverage Effective tools –
    training and delivery, input and output

FOUNDATION

Anchor in your
beliefs and values.

Your org’s learning
philosophy informs your L&D strategy.

  • Gather input from stakeholders
  • Map out what learning looks like
  • Write a philosophy statement
    (who, what, why)
  • This is high level – no perfect
    formulas for this
  • What does learning look like and
    what does it mean in your organization?

Core values:

  • How do you integrate these values
    into your culture?
  • Incorporate these values to
    inform training focus and to elevate the employee experience

The Employee
Experience:

  • Journey mapping
  • What does it feel like to work
    here?
  • How can we create an environment
    where people can truly grow?
  • Where can we partner across the
    business as reinforcement?

L&D Program is
really about change management – a form to drive change. Think about:

  • Opportunities
  • Strategy
  • Culture Shift
  • Feedback Loops

Uncovering Opportunities:

  • Use results from annual and pulse
    engagement surveys
  • Look at performance metrics –
    review SLAS – to identify gaps around skills/competencies
  • Get stakeholder feedback –
    informal and formal to round out data points

Create an Informed Strategy:

  • What are our biggest challenges?
  • Prioritize based on those that
    will have the greatest impact
  • ID groups that are most impacted
  • Build from competencies
  • Automate training
  • “This is how the new L&D
    program is going to support…”

Culture Stuff:

  • Model enthusiasm – leverage
    positive psychology, mirroring, cascade across and down all levels of
    leadership
  • Overt communications
  • Shared accountability – tracking
    with targets to meet

Feedback Loops:

  • Formal tools to get employee
    feedback
  • Leverage managers for informal
    feedback and formal
  • Leaders to demonstrate the new
    behaviors (modeling) and do check-ins with teams

Tools:

Your LMS,
facilitators, surveys, what they called “elevators” (ways to elevate
the experience – whiteboards, polls, gamification of sessions).

For compliance
training – if you’re trying to actually change culture, think about how these
behaviors ladder up to your values.

For leaders/managers
– provide them with a check-in template (for meetings, conversations)

When change happens,
values usually don’t change. But execution does chage. Ask “How do we
define what it means to exhibit our values?”

The Hitchiker’s
Guide to AI

Matt Donavan, GP
Strategies

Start first with
what the work is you need to do (not the tools, etc)

Put up Enterprise
Guard Rails:

  • Responsible Use of AI
  • AI Policy
  • Approved Tools

Guidelines for
responsible use of AI:

  • Maintain privacy and security
    (how identifying data being used)
  • Ensure safety and well-being
    (provide guidance)
  • Embrace DEI (how well does the AI
    work in different regions of the world 0 does it worry the same in Japan
    and Brazil….
  • Drive accountability (is the
    content generated accurate, do you have the rights for copy and or images
    being managed. Does it meet quality standards?
  • Convey Transparency: let people
    know when you’re using it – don’t be ashamed, but open.
  • Promote Fair and Equitable: does
    it bring people together. Does it drive equity?

GPT4 sitting in an
azure environment. Put all of your past RFPs on your system – point it to
meaningful data from historical RFPs. (but remember most org’s data is going to
be a hot mess – may require a lot of mopping up afterwards).

Your policy might be
“don’t put high-risk data onto that…”

Training Data Set
Through 2020 (?)

Constitutional
Training (how you set these ecosystems up).

Need both human
guidelines in place and the how we’re structuring machines

How can AI be used
to support a manager in the timeline completion of quality of performance
conversations?

Performance
Management Conversation Workflow….where could AI support this workflow?

  • These are the tasks in the
    workflow: Remember (to do it!), schedule, locate prior documentation, prep
    or refresh, coaching or mentoring, conducting the conversation, complete
    the documentation
  • Could you have AI help with those
    sticky spots? AI could check if you’ve scheduled it. It could go out to
    your system and get feedback from people, then summarized the people –
    then it locates the docs for you, then brings in other data and info.

Morning Buzz – eLearning – what do people
really want and need?

Cammy Bean, Kineo

This was an informal conversation about learner needs – and
the reality of what’s going on in the real world.  Individuals and orgs are at all levels of
maturity in terms of elearning. Lots of Storyline use. People leveraging
YouTube videos into course content (why recreate what’s already out there and
that people really want to see anyways – first thing most people do is Google
It or go to You Tube to learn).

Thursday Keynote:
The Five Mindsets of Innovation

John Linkner

Business leaders
need to be like jazz musicians

Adapt, navigate
uncertainty, situation awareness active listening, real-time collab and problem
solving, recovering  quickly from
setbacks.

Mundita – venture capital firm with a soul (this is his VC company)

Everyday Innovations

 

  1. Start before you’re ready

Awaken the artist
within

How have I always
done it in the past. What would it look like if you did the opposite  the Judo flip.

Savannah Bananas

Reduce risk through experimentation.
Lots of little ideas that you test – 15 minutes and 20 bucks fast. Cheap and
fast.

Create a To-Test
list – like a to do list. Keep a running list of things to try and test – the
mere existence of the list will boost your innovation. Do a shared list.
“What are we doing to test this week.” Inject an experimentation
mindset.

 

  1. Break it to Fix it.

Start with  a blank page – Jerry Seinfeld throws out all
his material every year.

 

Let go to move ahead. A hallmark of
innovators. Release an existing system to discover a new one.

 

At Ben & Jerry’s when a flavor is
no longer successful, they physically/literally BURY IT. They have a graveyard.
They give it a champagne toast and read a eulogy. The let it go. It’s ok to let
go to move ahead. And they have a digital version on their website. It’s ok to
break it to fix it.

 

Try this….the business funeral. Look
to a system, process, technique, approach that yo’ve been doing for decades.
Give it a funeral and a cake and let it go. We spend too much time trying to
save the past.

 

Speedo – break it to fix it – it used
to be the fastest suit was smaller, smoother. Then they made a sharkskin
suit…full body.

 

Try this…speedo did the judo flip.
Instead of small and smooth it was large and not smooth.

 

Shatter conventional thinking

 

  1. Use every drop of toothpaste:
    MacGyver It

 

Try this….print an image of a tv on
your bike box to reduce damage by 70%


4. SEEK THE UNEXPECTED

 

Before choosing A, B or C – think if
there’s an Option D or an Option X.

 

Option X is what are the obvious
answers and what are those far out answers, the unexpected approach.

 

Pittsburgh Children’s Hospital wanted
to create a better experience – they outfitted the window washers in super hero
costumes. Little innovations, big breakthroughs. This cost virtually $0 and
made a huge difference.

 

Bring those ideas to the surface. Find
the oddball idea that makes all the difference in the world.

 

  1. Fall seven times, stand eight.

 

Setbacks are part of
the process, not shameful – a learning opportunity. Each time, try something
new (don’t keep doing the same thing). Try a different creative approach.

100% chance with
jazz music that I’m going to screw it up. Confidence doesn’t come from
expecting to get it right, it’s from knowing how to bounce back.

Downtown Boxing Gym
(check it out)

 

Learning
Architecture Design: Anchoring ID/LXD in Business Transformation

Frank Nguyen,
Genentech (and formerly Amazon)

ID came about out of
WWII.

Robert Gagne, 1956
The Conditions of Learning – nine events of instruction – established the
beginning of instructional design.

Does training always
results in learning? No – sometimes.

Training is a tool –
sometimes a jackhammer – may be effective 
but not always so efficient.

Walk out of a room,
you’ll forget 42% of what you heard; and then if in 3 months you haven’t used
it it’ll go down way more. Ebbinghaus Curve was 1859 (?) – but results continue
to be replicated.

Gagne’s nine events
have really served as the basis of ID.

You might sit in a
class and follow all the nine events, it might still be really boring.
Following conventional instructional design can be really dull.

  • Early 90s – world
    wide web!
  • 2005 Wikipedia and
    Web 2.0
  • 2005 YouTube
  • 2007 iPhone
  • “Mobile
    Learning” people don’t even know what that is now – it’s just part of the
    world.

So how do we design
around this new world?

Instructional Design
+ User Experience Design = Learning Experience Design – ID has to evolve to
encompass ALL these ways of learning and working.

So does Learning
Experience Design always result in learning? No – maybe sometimes.

Do Learning
Experiences always achieve Business Goals or Transformation?

Instruction Design
> Learning Experience Design > are we due for the next evolution?

What is the problem
with LD and LXD?

Are we focused on
learning outcomes or business outcomes? Remember, training/learning may not
always be the solution.

These don’t move at
the speed of the business – the business needs much faster solutions

Immediacy,
application, tying it back to the learner.

Tenet #1:

Sometimes learning
happens because of training, but learning happens all the time with or without
us.

Learning produces
business outcomes. Run a learning organization and not a training organization.

Jane Hart’s list of
top learning tools:

YouTube, Google
Search, Teams, ChatGPT….then things like Zoom went down….there are no
“training tools” on the top ten list – a user focused list. Where do
you go to learn? Learning is happening all the time, with or without us.

How are you pulling
in youtube etc into your learning architecture?

Tenet #2:

Learning is just one
leg of the tripod for change management and business transformation

Change management
model: ADKAR

  • Awareness (Communication –
    lighter weight, less frequency – training class is a jackhammer when you
    need a screwdriver),
  • Desire (marketing & comms),
  • Knowledge (training),
  • Ability (workflow learning and
    experiential learning),
  • Reinforcement (workflow learning,
    incentives, performance management, marketing and comms)

How do you bring
value to people’s daily jobs…to drive real business transformation, we have to
think outside the training box.

Tenet #3:

Focus on learning
outcomes not business outcomes

Working Backwards –
Insights, Stories and Secrets from Inside Amazon (Colin Bryar and Bill Carr)

Solve real business
problems through learning, not just learning objectives

Working Backwards:

  • Who is the customer?
  • What is the customer problem or
    opportunity? What problem does the customer need to solve?
  • What does the customer experience
    look like?
  • Is the opportunity compelling
    enough?
  • Build a roadmap, working backward
    from the end-state.

You may need to have
a long perspective – it can take a long time to change.

“That’s where
we want to be and these are the iterative steps tto get there.”

Instructional Design
> Learning Experience Design > Learning Architecture Design
 (THIS is the current transformation)

Learning
Architecture Design = Much more high-level.

Learning Architect
Working backwards

  • Who is the customer (learner)?
  • What is the business problem (not
    the learning problem)?
  • What will be different in the
    business over next one, two, three years?
  • Design a learning architecture by
    working backwards from the end state
  • Anchor marketing, comms, and
    learning to business milestones or seasonality

Learning side will
need to be agile because the business may change.

Amazon fulfillment
center:

Jan – ramp down

Feb-May –
invention/continuous improvement

June – ramp up

July Prime Day
(created to be a test day for inventions)

Aug-Sept iteration

October ramp up

Nov-Dec peak

  • Align
    training/learning architecture with business seasonality

Think about:

  • Business Milestone
  • Marketing
  • Communicate
  • Workflow learning
  • Training (last
    resort)

Managers – consider
creating a 15 minute meeting in a box for an experiential learning exercise in
a team meting

If training/learning
is tied to your job level, pay, etc. you may pay attention/learn

https://frankn.net/DL23-WBT.pptx

 

 

Getting Started with Learning Content Anaytics 

Cammy Bean, Joe Replin, Rodrigo Bolanos, Kineo 

 

A man standing in front of projected screen share, pointing at numbers on a mocked up data dashboard.

This was our session! Quick story – analytics is top of mind for L&D but it’s a low skill set on most teams. Many orgs are just trying to figure out how to get started and there’s a lot of overwhelm. We wanted to set the stage for why data & analytics are so crucial to L&D, share a framework for getting started with building these skills by looking more closely at learning content analytics, and share some examples and case studies from some of our work with different types of organizations.

Our deck is up on slideshare for your viewing pleasure.

 

Harnessing
the Power of AI to Develop Scenarios

 

Jennifer
Foster, Udutu

 

Using AI
tools to jump start your workflow and generate branching scenarios:

 

  • Use ChatGPT4 ($20 a month) – to
    generate list of questions to ask your SME about X topic.

 

 

 

  • Scenario Builder Epic ChatGPT
    (based on Cristy Tucker’s branching scenario work)

 

 

 

 

 

  • Zoom just came out with an AI
    tool last week – it transcribes your whole meeting for you and then throw
    that into ChatGPT – “hey give me my action items from that call”

 

 

  • Get around the uncanny valley
    feeling about avatars by just telling your learners upfront – “hey,
    this was created with AI”.

 

  • Research show: AI created avatar
    videos match human presented ones in effectiveness

 

 

 

Making Your L&D
Case by Storytelling with Learning Data

Eli Bendet-Taicher,
Wix

Think about the
story you want to tell – think about different ways to tell that data story –
two to three ways (each version will vary depending on where you put your hook,
your facts, etc.)

What do you want
your audience to…?

  • Know – facts data
  • Think – perception
  • Feel – how do you want them to
    feel about the problem
  • Do – what do you actually want
    them to do after the session?

“Data in
L&D is a lot like sex….everyone is talking about it, but not many are doing
it, or even doing it well.”

First party data:

  • What can I collect myself? (e.g.,
    directly from your LMS if you’re in L&D, etc.)

Second party data:

  • What kind of data do other
    departments collect within the organization that I can leverage?
  • How can I get my hands on second
    party data?
  • Second party data – sales,
    revenue, engagement surveys, performance reviews, business roles
    performance, quality control

Ideally, you want to
correlate first party with second data to show impact

  • Reaction = first
    party data
  • Learning = first
    party data
  • Behavior Change
    =  first and second party data
  • Results = second
    party data

ROI (financial
metric) vs. ROE (indicator of value

ROE – measurement of
qualitative and quantitative benefits – did you meet expectations?

Correlating L&D
Data with KPIS – the flow.

  • What are your organizational
    business KPIS
  • How does L&D support the
    business KPIS
  • Derive your own KPI from business
    KPIs
  • Collect data
  • Correlate all data points
  • What kind of story does it tell
    you?
  • Communicate that story to
    stakeholders

We remember numbers
when the numbers serve the story (he showed a pitch video and the guy shared a
lot of numbers) – when you tell a story with data, think about the story you
want to tell (replace your adjectives with numbers — a lot, again and again,
any, not a lot-huge, game-changer, surprising – can probably all be replaced
with numbers.)

Use infographics

Tell the data story
and PR your work.

Closing Keynote: Michelle Poler

Michelle Poler, author of Hello
Fears and survivor of her own “100 Days Without Fear Experiment”

This was a feel good
session that literally had the crowd dancing by the end. Michelle shared
stories of facing ones fears, being clear on what you want, and making a shift in
your life to live more authentically and courageously.

The Expo Hall

150+ vendors – biggest year ever

BIG booths like DevLearn has never seen before!

·       Articulate
– showing off AI power that’s coming soon to a Rise course near you (upload
your content docs and press a button and voila – a course! Click a button to
translate your content. Includes Video Avatar creator).

·       Regis
– showed off an AI driven branching scenario builder.

·      
Note – both Articulate and Regis were a
bit smoke and mirrors as I suspect all developers were working until the day
before the conference to get things together. This is all happening in REAL
TIME.

·       Big
LMS booths from Docebo, Schoox.

·       SmartCat – for
AI driven translations

OTHER

And then there’s all the conversations in the halls and
over dinners and captured in passing.

Lots of interesting things happening with AI driven
chatbots and agents – this will change how people search for and access content
– looking for answers when they need it in the workflow.

AI will change our workflow – how we do our everyday tasks.
This will put huge pressure on content companies to reduce cost and speed to
market. We need to pivot fast.

It will also change how learners search for and consume
content – Learner Experience will be less about a big 45 minute course and more
about that chatbot….of course, formal learning/training will still exist, but
it’s going to be different.

Obviously, all the concern about regulations and ethics and
privacy in any conversation on AI. While I don’t mention that much in my notes,
it’s an ever-present thread (thus some of the fear).

Lots of fear, uncertainty – but also a lot of EXCITEMENT
about the possibilities.

 

See you next year, #DevLearn!

Leave a Reply

Your email address will not be published. Required fields are marked *