Welcome to the SeeMeCNC Community

So, I didn't see anything mentioning this, but it could be I didn't dig well enough. I know the diameter of the nozzle is stated at 0.5 mm (This is the nozzle that came with my Rostock Max kit) but is this how big the filament being extruded out of it should be? As in, the diameter of the hole should be 0.5 mm? I checked mine just for grins and it seems the filament coming out of the nozzle is 0.6 mm when being extruded into air. Not sure if this is normal, but I need to know since I'm sure it makes a difference to the slicer, etc.

That is normal.

It's normal it's called die swell.
At some level it doesn't really matter, because filament is stretched and squashed when printing, you can print smaller than the free air extrusion width or bigger. The usual rule of thumb is you want extrusion width to be just a little larger than your nozzle diameter, so for a 0.5mm nozzle around 0.55 mm

Mine was 0.6mm for what is supposed to be a 0.5mm nozzle. I can freely pass a 0.5mm bit through the nozzle hole when it is cold. I imagine it expands a bit when hot. So should I keep my nozzle setting at "0.5mm" or change it?

You should run a .5mm single wall calibration cube with no top or infill. Measure the wall thickness. If the wall is thicker than the nozzle size set in the slicer, then lower the extrusion multiplier. if the wall is thinner, then raise the extrusion multiplier.

Ultimately you want the wall thickness to match the nozzle size in the slicer. Also, you should design the cube yourself. That's the only way to guarantee the wall thickness is what you expect.

Thank you KAS for the advice. I will give this a shot. Time to fire up AutoCAD!