Go big or go home?

A little bit about me

I work on explaining AI models.
And as someone working on explaining AI models, first I need to make an AI model.
This is the story of how I spent an year relearning how to make AI models

And learn where ✨luck✨ plays a part in making effective neural networks

Before we start

You’ll be asked a few questions
By responding, you agree to allow us to use your data.
Your answers
- are anonymous.
- cannot be individually idenitified.
- and only summary statistics are stored.
- are not marked and will not affect your grade.
You can choose not to respond to any question.
You will be able to use a web app during the session which does not collect any data.

Participating in these activities can help you build your understanding, and providing your responses will help us improve how we teach this topic.

You can learn more details about the project at https://bit.ly/4btsNrR.

Now back to the show

Fitting a neural network

It’s hard, right? We’d like to hit an easy button to get a model!
Most of the time, we care only about the performance of the model and the modeling pipeline.
Given the computational power we have, we rarely care about the size or complexity of the models.
If it predicts well all is ok, right?

Quick refresher

Playground

What is the role of a neuron in the hidden layer?

defines a logistic function
activates the net
fits one part of the data
removes noise
makes a linear cut

Do you think the random seed used will affect a fit?

yes
no
maybe

Will more neurons always result in a better fit?

yes
no
maybe

Join the online poll at menti.com with code 4398 8771

Let’s work on fitting this dataset

This has two predictors, x, y and

one response, Class, a categorical variable with two levels.

Neural network size?

Hyperparameters
Question

These hyperparameters are fixed for now

Number of epochs: 100
Batch size: 71
Loss function: Binary Cross Entropy loss
Optimizer: Adam
Training set size: 5,000
Testing set size: 5,000

How many neurons are needed in the hidden layer?

How many neurons?

Join the online poll at menti.com with code 4398 8771

Let’s try some different data and NN sizes

We will fit two models: 5, 30 neurons
We will also re-fit multiple times

For that we will use a special tool to rerun these fits multiple times

Your turn

Go to this app

random-nn-playground.janithwanni.com

Play with setting the number of neurons and re-fitting multiple times.

Pay attention to model size

Think about how the fitted model looks with the different choices.

Start running

You’ve got 10 minutes

What have we learned?

The larger neural network has more consistent fit statistics
The smaller neural network is as accurate as the larger one
- sometimes
- but has a lot of variability in the model fit (depending on the initial random seed)

QUESTIONS for you

Q1
Q2
Q3
Q4
Q5

What is the role of a neuron in the hidden layer?

defines a logistic function
activates the net
fits one part of the data
removes noise
makes a linear cut

Do you think the random seed used will affect a fit?

yes
no
maybe

Will more neurons always result in a better fit?

yes
no
maybe

In one word how would you describe this session?

How would you rate your experience in this session?

The application worked smoothly
I enjoyed trying out different model fits by various seeds
I would recommend this application to other students

Join the online poll at menti.com with code 4398 8771

Wrapping up

Be curious
Care about the little things
Ask why and how things work, or do something different from what we expect
The big black boxes of neural networks are not so dark with little glimmers of light

Thank you!

Have any suggestions or ideas?

The colour palette for these slides are inspired by the photograph by Bill Henson as part of the art installation Oneiroi in the Hellenic Museum, Melbourne.

Janith Wanniarachchi

janith.wanniarachchi@monash.edu

@janithwanni

@janithwanni.bsky.social

janith-wanniarachchi

janithwanni.com