Minimal Raspbian distribution

Packages that should be installed:

libraspberrypi-bin
raspberrypi-bootloader
raspi-config

List of packages to remove has been moved to a blog post: Slimming an existing Raspbian install

Building an image from scratch

Buildroot has instructions for an Raspberry Pi target.

https://github.com/debian-pi/raspbian-ua-netinst: Raspbian network installer

ansible

https://github.com/escapologyBB/ansible

https://github.com/escapologyBB/ansible-raspberry-pi

http://jpmens.net/2012/06/06/configuration-management-with-ansible/

https://github.com/geerlingguy/diet-raspbian

Etc

Instantly finding a headless Raspberry Pi