The orange thing is for subscribing to this website with RSS. I don't have any idea what RSS and "feeds" are, either.
It might stand for Really Simple Syndication, or it might stand for Rich Site Summary, depending on who you ask. But I found a place that makes it really easy to understand. It's called Feedburner. It's really friendly and talks to you in a nice, conversational tone.
Basically, a feed is a way to subscribe to any number of your favorite websites that update their content frequently. According to "Feed 101," an article at their website, feeds help you "streamline your online experience by subscribing to specific content feeds and aggregating this information in one place to be read when you're ready." Which vaguely means something to me, sort of.
Anyway, they have made it really easy and all you have to do is click the big orange thing below to get started. Once you're there you should be able to figure it out. Also below are automatic "add" links if you happen to use a popular web-based feed reader already. If it's not there, more auto-add buttons are available at the Feed page. Thanks for your interest!
If you're up on all this stuff already and want to bypass Feedburner, the actual XML feed is located at /blogfeed.xml