I don't even read any assurances from Rogers, just weasel words like what the NSA says:
Nest's founder Matt Rogers anticipated this issue, and insisted that there's no reason to worry. In his blog post, he says that data won't be shared with anyone, including Google, without a customer's permission.
Probably in the fine print that by using the product you have given permission, and if you don't want to give permission you need to opt-out by sending a letter through the USPS. Oh, and it must be received on a day when there is a full moon, signed in blood, and sealed with the king's seal.
In this day-and-age of corporate spying you must always assume the worst.