A "hardware register" might also refer to a location inside some hardware device. For example, a UART (COM port) looks like a D-shaped connector with 9 or 25 pins from outside the cabinet, but to the device driver it looks like several configuration registers, a status register, and data registers holding the next character to send and the last character received. (I've left out a *lot* of detail there.)
In the x86 architecture those registers are usually located in a special physical address space that is accessed with I/O instructions. In other platforms, it is common for hardware registers to be mapped to some corner of the normal memory space. In either case, one of the important roles of an operating system and its device drivers is to prevent application code from needing to know the details of where the hardware registers are located and what they mean.
In some kinds of hardware devices, the distinction between memory and hardware registers is less clear. For instance, your video adapter contains a block of memory known as the frame buffer that holds the color and brightness values for each individual pixel. Is that memory a large hardware register or is it just a buffer that has an interesting side effect?