Holy cow, I wrote a book!
You may have noticed that the numerical value of the
DS_SHELLFONT flag is equal to
DS_FIXEDSYS | DS_SETFONT.
DS_FIXEDSYS | DS_SETFONT
#define DS_SETFONT 0x40L /* User specified font for Dlg controls */
#define DS_FIXEDSYS 0x0008L
#define DS_SHELLFONT (DS_SETFONT | DS_FIXEDSYS)
Surely that isn't a coincidence.
The value of the DS_SHELLFONT flag was chosen so that
older operating systems (Windows 95, 98, NT 4) would
accept the flag while nevertheless ignoring it.
This allowed people to write a single program
that got the "Windows 2000" look
when running on Windows 2000 and got the "classic" look when
running on older systems.
(If you make people have to write two versions of their program,
one that runs on all systems and one that runs only on the newer
system and looks slightly cooler,
they will usually not bother writing the second one.)
The DS_FIXEDSYS flag met these conditions.
Older systems accepted the flag since it was indeed a valid flag,
but they also ignored it because the DS_SETFONT flag
This is one of those backwards-compatibility exercises:
How do you design something so that it
is possible to write one program that
gets the new features on new systems while at the same time
degrading gracefully on old systems?