In Make magazine, Vol. 49 March 2016, they mention the C.H.I.P, this is a $9 single board computer, though it is not yet shipping, and is scheduled to start shipping November 2016.
There is also the Adafruit Trinket - Mini Microcontroller - 5V Logic $6.95. Full details listen at linked site. Information on page does state to use the 3V version if you intend to run on LiPo batteries, and for devices that require 3V logic.
Also there are a couple of listed tradeoffs for this unit, again from the units data page:
Trinket does not have a Serial port connection for debugging so the serial port monitor will not be able to send/receive data
Some computers' USB v3 ports don't recognize the Trinket's bootloader. Simply use a USB v2 port or a USB hub in between.
Also states that while the Trinket is programmed via the Aduino IDE, it is not 100% Arduino compatible.
I hope this answer helps.
EDIT Oct 20, '16: I failed to notice that the Trinket does not seem to have internet access.
There is a tiny programmable wifi card ESP8266 So you can add wireless capability to devices without built in LAN/Wifi connectivity.