Thanks to Roman for this hint:
If I use the usual text based terminal console on a HP DL180G6, on the iLO I just have some cryptic writings. Since the iLO can display graphical content (and it seems that this can be done better than text content), switching the VGA mode to a graphical mode works.
To do that, just append in Grub (menu.lst) a VGA=<VGAmode> to the kernel options. For example:
vga=0x314 (800x600 with 16bit colors)
vga=0x300 (600x400 with 8bit colors)
See here a list with available VGA codes.
Update: Since Grub2, on Ubuntu you need to add the values into /etc/default/grub and run update-grub afterwards.