Harper's shared items in Google Reader

Your grocery-store data, shared

2008-07-04T22:12:00Z

Get-rich tip: Be the first grocery store to provide a digital purchase history to customers, for use in household planning.

This would actually convince me to use a frequent-shopper card and provide accurate information for it. Every time I made a purchase, a record (in some open standard format) would be emailed to an address of my choosing. In this fashion, I could build a household food database.

A sufficiently sophisticated grocery chain could even make the purchase history available via Web app, though the email option should still be there so other tools could be used.

An ambitious customer could even use a handheld scanner at home on items as they are thrown away. This would allow you to have an accurate, available-from-anywhere database of what is in your fridge and/or pantry. A scanner would also allow you to scan in purchases from other retailers (i.e. wine from your local wine shop, cheese from the cheese shop, etc).

And of course you could enter data in manually. With the right interface, this wouldn't be such a pain. With one click, for example, you could record the repurchasing of anything you've ever bought before.

If a bar-code scanner were built into an iPhone or Blackberry, one could quickly check whether a particular item is already in the cupboard at home, or even find out if one is being charged a reasonable price.

Independence Day

2008-07-04T18:59:44Z

My young and lovely wife, showing what might be overoptimism or maybe artful timing given the economy but more likely just general disappointment with me, has decided to embark on a career in real estate sales. She has taken classes and passed tests, joined one of the very best local firms, and hurled herself into the business of selling historic Charleston homes while they still have some value and the termites haven't finished their work. And along the way, while mastering the Multiple Listing Service, she learned an important fact that was news to us both: people no longer find houses for sale by looking in the local newspaper. They use the Internet, instead.

The irony here is that -- at least in these parts -- the local paper seems chock-full of real estate ads. But according to her teachers down at the MLS university, those listings are simply vestigial, like little toes we all have but probably don't need for balance or, indeed, for anything at all. Real estate brokers put ads in local newspapers because their customers expect them to do so, not because they actually help sell houses.

I'm sure there are exceptions to this rule, but if 80 percent of all houses for sale in the U.S. are eventually sold NOT because of any newspaper listing, tradition or professional pride aside, at some point we can expect real estate newspaper advertising to eventually disappear. Chock up more bad karma for the newspaper industry, where this fact has to have been long known, and which is apparently in even worse trouble than we thought.

But this column isn't about the newspaper industry or even about the real estate industry. It is about the lack of friction in our commercial lives brought about by the Internet and an emerging thought in my mind that maybe it is time we as a people took action to change some things.

Let me explain.

It's not that newspaper ads work so poorly for selling real estate, it's that Internet advertising works so well. You can put more words on a web ad than you could ever put in the newspaper for the same money. You can put more and bigger pictures, virtual tours, Google maps. You can put Zillow virtual appraisals and links to lenders, home inspectors, and the local Chamber of Commerce. Internet house listings can be searched in a zillion ways that newspaper listings cannot. In the time it takes to find a house -- any house, maybe even the wrong house -- in the newspaper and then go see it, well in that amount of time using the Internet you can find the house, order an inspection, get a loan, and make an offer on the darned thing. It's like crossing house-hunting with air hockey.

But is it all good?

Don't tell George W. Bush, but we are in a recession, which is making me look more critically at the Internet as a marketplace. There's a lot of good about the Internet market, of course. Auction sites like eBay help us get rid of our junk and then help us replace it with new junk. The web has made comparison-shopping for houses and cars and disposable diapers almost a contact sport. And we're sure as heck better equipped than we were before to claim all that money that's been waiting for us with some bank manager in Nigeria.

Just as an aside, I know a guy from Japan who actually went to Nigeria once to pick up some of that unclaimed money. It didn't exist and he felt lucky to get home at all.

The theme of disintermediation -- of eliminating middlemen -- has been a driving force in the Internet for as long as commerce has been allowed on the web. But what happens when the middleman you just eliminated had as one of his or her jobs the task of keeping us from being ripped off?

Tasks that are harder to accomplish are also less likely to be foolishly accomplished, which is why so few of us make trips to Nigeria.

That's not the way we are supposed to view things, of course. Ideally the Internet as a research tool is supposed to give us all the information we need in order to resist any allure the Internet has as a tool of fraud or misadventure. But this attitude ignores many of the fundamental forces at work in most sales situations where the simple fact is that we want to buy, the seller wants to sell, and so any countervailing forces are purely voluntary, which is to say often nonexistent.

Take our current national economic mess, the so-called sub-prime mortgage crisis. I like to think that I'm not a subprime kind of guy, but pretending to work as I do (my kids think I TYPE for a living) the world may not always see me the way I would like to be seen. So last year, in what we didn't know were the waning and idyllic pre-subprime days, I tried to get a new mortgage. Of course I used the Internet to get the loan because, as we all know, when banks compete I win. And within a few days, without having to actually meet with or even speak to another human, I found myself offered a $336,000 mortgage.

It was SO easy. Fill out a few online forms, make some choices, and there I was, about to close that loan. But then I did an odd thing. I carefully read the papers I was about to sign (I'm one of THOSE people). And in that residential loan application, right on line something or other, was a number that didn't make any sense to me at all. It was labeled "total household income" and was almost twice the pitiful amount I actually earn.

From where did that number come? It certainly never came from me. Since my signature would be at the bottom of this application I wanted to make sure everything was correct, so I called the mortgage broker. For the first time we spoke. She was a very nice lady, too, and explained that number was the variable required for all the ratios to be correct so I could qualify for the loan.

"But it isn't true," I said.

"Do you want the loan or not?" she asked.

Not.

I wasn't so principled as cowardly, but maybe that doesn't matter: I did what I knew was the right thing for me, which was to walk away from the loan. But evidently a lot of other people took the other course and today are having trouble paying for their houses, which is a big part of the reason why we are in this current economic mess.

This little drama of mine explains the credit crunch better than Federal Reserve chairman Ben Bernanke ever would. Securitization of mortgages works just fine unless the mortgages are based on lies. Lenders turned a blind eye to bad loans and bad loan candidates because another company assumed the risk by bundling these loans and reselling them on a global market.

What has caused the credit problems to extend beyond subprime borrowers to just about everyone is the simple fact that lenders can't act so sloppily now, but having turned that blind eye for so many years they have no idea who is telling the truth anymore. So they don't trust anyone.

And that brings me back to transparency and disintermediation and why the heck the Internet, which was very involved in enabling a lot of this bad behavior, didn't do even the smallest thing to help save us from ourselves?

I suppose it was because there is no money in virtue, no easily measurable value in NOT having those banks compete so I could win only to eventually lose.

Do these loan referral outfits like LendingTree and LowerMyBills and the many, many others EVER say, "Wait a minute, pardner, there's no way you can qualify for any loan, much less that no-doc super-jumbo you have your eye on?"

No.

In their defense, these companies are never actually faced with that question, which is ultimately asked not of them but of their customers, the lenders, and we know how much self-restraint those people have: almost none.

Here's why I bring this up. It is clear to me that government (ANY government, not just the U.S. federal government) and Wall Street have no idea whatsoever how to handle the current crisis. They are just trying to look busy while protecting their own interests and allowing those affected to muddle our way through this mess to some kind of solution. It's not that they don't want to be helpful (if the cost of being helpful is low enough) but that they simply don't know HOW to be helpful. They can't be educated and they can't be changed. Certainly they wouldn't consider any course that would curtail government authority or commercial opportunity.

So I figure we're on our own. And if we are really, truly on our own, we shouldn't pretend that we're not, that some agency that doesn't know its IP address from a hole in the ground will take care of us and make this all better. If we're on our own we should solve our own problems using the tools at our disposal. Which brings me back to the Internet, where it ought to be possible for a change to use all that transparency and economic friction reduction to actually do something FOR us, rather than something TO us.

So where is the next wave of financial start-ups that view ME, not Citibank, as the customer?

Another favorite word from the 1990s was "disruptive." Your start-up needed a disruptive technology or a disruptive business model -- anything to throw the market on its ear and allow your start-up to accumulate market share before the incumbents figured out how to compete. But nearly all such disruption, at least the disruption that survived the Internet meltdown of 2001 and therefore was based on real -- rather than voodoo - economics, was on the sell side. It was companies finding new ways to take our money.

There are few really disruptive technologies or business models on the buy side, but one that stands out is Craigslist, which is close to unique in its efficacy, impact, and the fear it has put into an entire industry (newspapers, bringing us full-circle, see?).

I'm not asking for a revolution, just 2-3 more Craigslist-type successes that actually put consumers first and don't just say they do while selling our identities out the back door to some marketing mafiosi.

Where the Super Bowl-advertising dot-coms of the 1990s didn't know and didn't care where their profits were coming from, the dot-coms of today are obsessed with profitability and the easiest way to make a profit is from saps like me. I'd like that to change, please.

But not all news in this area is bad. Sometimes unscrupulous behavior gets what it deserves. In Australia, for example, eBay just tried to make PayPal not just its preferred payment system for auctions, but the ONLY payment system eBay Australia would accept. That's seeing eBay customers not as customers but as sheep to be sheared. Fortunately the results of this bullying were disastrous for eBay Australia, which is swooning as customers bail for other auction sites that are less greedy or maybe just see themselves as less powerful. It's a powerful lesson for eBay that may cost them Australia and hopefully will teach them a lesson about REAL customer service.

It's a sign of the times, or maybe I just hope it will become one.

We're mad as Hell and we aren't going to take this anymore!

A breakdown of what Viacom was granted and denied in the recent ruling in its case against YouTube.

2008-07-03T18:04:00Z

Because I hadn't seen one yet, I thought I'd compile a small breakdown of what Viacom asked the court to order Google to reveal - along with some excerpts of the ruling.

The breakdown:

1) The source code for web search. Denied, protected.

Plaintiffs move jointly pursuant to Fed. R. Civ. P. 37 to compel YouTube and Google to produce certain electronically stored information and documents, including a critical trade secret: the computer source code which controls both the YouTube.com search function and Google’s internet search tool "Google.com".

Plaintiffs argue that the best way to determine whether those denials are true is to compel production and examination of the search code. Nevertheless, YouTube and Google should not be made to place this vital asset in hazard merely to allay speculation. A plausible showing that YouTube and Google’s denials are false, and that the search function can and has been used to discriminate in favor of infringing content, should be required before disclosure of so valuable and vulnerable an asset is compelled.

2) The code behind YouTube's identification of infringing videos. Denied.

Plaintiffs also move to compel production of another undisputed trade secret, the computer source code for the newly invented "Video ID" program. Using that program, copyright owners may furnish YouTube with video reference samples, which YouTube will use to search for and locate video clips in its library which have characteristics sufficiently matching those of the samples as to suggest infringement.

The notion that examination of the source code might suggest how to make a better method of infringement detection is speculative. Considered against its value and secrecy, plaintiffs have not made a sufficient showing of need for its disclosure.

3) Copies of all removed videos. Granted.

Plaintiffs seek copies of all videos that were once available for public viewing on YouTube.com but later removed for any reason, or such subsets as plaintiffs designate (Pls.’ Reply 41).

While the total number of removed videos is intimidating (millions, according to defendants), the burden of inspection and selection, leading to the ultimate identification of individual “works-in-suit”, is on the plaintiffs who say they can handle it electronically. Under the circumstances, the motion to compel production of copies of all removed videos is granted.

4) Logs data including the "Login ID" and the IP address for each view of a video on YouTube. Granted.

Defendants do not refute that the "login ID is an anonymous pseudonym that users create for themselves when they sign up with YouTube" which without more "cannot identify specific individuals" (Pls.’ Reply 44) , and Google has elsewhere stated:

We . . . are strong supporters of the idea that data protection laws should apply to any data that could identify you. The reality is though that in most cases, an IP address without additional information cannot.
Google Software Engineer Alma Whitten, Are IP addresses personal?, GOOGLE PUBLIC POLICY BLOG (Feb. 22, 2008), http://googlepublicpolicy.blogspot.com/2008/02/are-ip-addresses-personal.html (Wilkens Decl. Ex. M).

Therefore, the motion to compel production of all data from the Logging database concerning each time a YouTube video has been viewed on the YouTube website or through embedding on a third-party website is granted.

5) Metadata for every YouTube video including titles, keywords, comments, flags, poster's username, and other administrative information. Denied.

No sufficiently compelling need is shown to justify the analysis of "millions of pieces of information" sought by this request, at least until the other disclosures have been utilized, and found to be so insufficient that this almost unlimited field should be further explored.

Therefore, the motion to compel production of all those data fields which defendants have agreed to produce for works-in-suit, for all videos that have been posted to the YouTube website is denied.

6) The schema for Google's advertising databases. Denied.

However, given that plaintiffs have already been promised the only relevant data in the database, they do not need its confidential schema (Huchital Decl. ¶ 8), which "itself provides a detailed to roadmap to how Google runs its advertising business" (id. ¶ 9), to show whether defendants were on notice that their advertising revenues were associated with infringing videos, or that defendants decline to exercise their claimed ability to prevent such associations.

7) The schema for Google Video's databases. Granted.

Plaintiffs argue that the schema for that database will reveal "The extent to which Defendants are aware of and can control infringements on Google Video" which "is in turn relevant to whether Defendants had 'reason to know' of infringements, or had the ability to control infringements, on YouTube, which they also own and which features similar content." Id. 52 (plaintiffs’ italics). That states a sufficiently plausible showing that the schema is relevant to require its disclosure, there being no assertion that it is confidential or unduly burdensome to produce. Therefore, the motion to compel production of the Google Video schema is granted.

8) Copies of all of the videos on YouTube marked "private". Denied.

Defendants are prohibited by the Electronic Communications Privacy Act ("ECPA") (18 U.S.C. § 2510 et seq.) from disclosing to plaintiffs the private videos and the data which reveal their contents because ECPA § 2702(a)(2) requires that entities such as YouTube who provide "remote computing service to the public shall not knowingly divulge to any person or entity the contents" of any electronic communication stored on behalf of their subscribers and ECPA § 2702 contains no exception for disclosure of such communications pursuant to civil discovery requests.

8) All non-video data regarding videos on YouTube marked "private" including the number of times watched or embedded. Granted.

Plaintiffs need the requested non-content data so that they can properly argue their construction of the ECPA on the merits and have an opportunity to obtain discovery of allegedly infringing private videos claimed to be public.

There's some things I like about the ruling (of course Google's search source code shouldn't be handed to Viacom) but I'm sad about the concerns about release of user data being considered "speculative." Is this a legal definition separate from the normal usage of the word? Because I can show pretty easily that usernames are often not "anonymous pseudonyms" and that many people use their full names. Linking video habits to a specific person wouldn't be that hard, particularly for those who played by the rules and are content producers that use YouTube promotionally and used their full names and have public profiles linking to websites, blogs, etc.

I'm sad about the IP address arguments as well. I understand Google's in a tricky spot here but the argument they made that IP addresses are "in most cases" not identifiable has been conflated by the court to mean that IP data "cannot identify specific individuals." That's false. People hosting web sites from static IP addresses where they also use the internet (e.g. some small businesses) can be identified by their IP data. Because there's fewer of these cases means that the data can be handed over?

Also, getting all logs data just because a claimant suspects infringement seems too broad. Why not just number of times viewed during various time periods? Does this mean I should ask MTV Networks for demographic data they've collected for all of their content since they've used songs from one of the bands I've been in and I suspect they haven't told me about all the times it was used and aired? Was it just that one time on the Ashlee Simpson show? Really? How can I be sure unless they hand over all usage data, related or not? Also, I'd like all of their advertising data so I can see if I was treated fairly in terms of compensation. I would love that data! I promise I wouldn't use it as an advantage in creating a new business.

Second-Class Google Citizens

2008-07-03T18:32:00Z

Every time Google releases a new feature for Gmail, Calendar, Google Docs etc. people who use Google Apps are left wondering whether they'll get the new feature. Sometimes they have to guess addresses, like for the new mobile Google Talk. They are supposed to figure out that the Google Apps version of: http://talkgadget.google.com/talkgadget/m is http://hostedtalkgadget.google.com/a/YOURDOMAIN.COM/talkgadget/m.

Besides having to deal with delayed updates and mysterious addresses, Google Apps users usually have at least one standard Google account and it's difficult to switch between the two parallel worlds. I noticed that when you go to Google Sites and you're logged using both a Google account and a Google Apps account, you are asked to choose one of them:

Maybe Google could somehow integrate Google accounts with Google Apps accounts so you can access all the services and get all the new features. The services that are part of Google Apps should have a customized interface and functionality, while the other services should only interact with them so you can, for example, share Google Reader items with your Google Talk contacts.

Viacom Wanted the Source Code for Google's Search Engine, But Obtained YouTube's Server Logs

2008-07-03T22:22:00Z

In the ongoing trial between Viacom and Google, regarding the videos uploaded to YouTube that infringe Viacom's copyright, Viacom really wants to prove that the most popular videos watched at YouTube were from its programs. Viacom even claimed that Google's search results are biased to give better ranking to the infringing YouTube videos, so it asked for... Google's source code (and YouTube's source code too). Here are some excerpts from the rulings:

Plaintiffs move jointly pursuant to Fed. R. Civ. P. 37 to compel YouTube and Google to produce certain electronically stored information and documents, including a critical trade secret: the computer source code which controls both the YouTube.com search function and Google's internet search tool "Google.com". YouTube and Google cross-move pursuant to Fed. R. Civ. P. 26(c) for a protective order barring disclosure of that search code, which they contend is responsible for Google's growth "from its founding in 1998 to a multi-national presence with more than 16,000 employees and a market valuation of roughly $150 billion" and cannot be disclosed without risking the loss of the business.

The search code is the product of over a thousand person-years of work. There is no dispute that its secrecy is of enormous commercial value. Someone with access to it could readily perceive its basic design principles, and cause catastrophic competitive harm to Google by sharing them with others who might create their own programs without making the same investment. Plaintiffs seek production of the search code to support their claim that "Defendants have purposefully designed or modified the tool to facilitate the location of infringing content." (...) YouTube and Google maintain that "no source code in existence today can distinguish between infringing and non- infringing video clips -- certainly not without the active participation of rights holders".

Unfortunately for Viacom and Google's competitors, the request to provide the source code has been rejected. But another request, this time for YouTube's server logs, has been approved.

Defendants' "Logging" database contains, for each instance a video is watched, the unique "login ID" of the user who watched it, the time when the user started to watch the video, the internet protocol address other devices connected to the internet use to identify the user’s computer ("IP address"), and the identifier for the video. That database (which is stored on live computer hard drives) is the only existing record of how often each video has been viewed during various time periods. Its data can "recreate the number of views for any particular day of a video." Plaintiffs seek all data from the Logging database concerning each time a YouTube video has been viewed on the YouTube website or through embedding on a third-party website. They need the data to compare the attractiveness of allegedly infringing videos with that of non-infringing videos.

Google argued that the task requires a lot of resources, since the logging database has 12 TB, and it violates users' privacy. Google has previously stated in a blog post that an IP address without additional information cannot identify people, so it's not personal information. "Therefore, the motion to compel production of all data from the Logging database concerning each time a YouTube video has been viewed on the YouTube website or through embedding on a third-party website is granted."

Viacom wanted other things: the schema for Google's advertising database and for Google Video's database, data about private YouTube videos etc. You can read the entire document as it's pretty entertaining.

Salon thinks that "all's not lost. Google might manage to reverse this decision on appeal, and Viacom, gauging the outrage, could decide to withdraw or limit its request." After all, getting YouTube's server logs just to determine the popularity of the infringing videos is an abuse: YouTube could have offered aggregated data about those videos.

Update: According to Search Engine Land, Google sent a letter to Viacom regarding the removal of personal data.

Given Plaintiffs' stated reasons for seeking information from the logging database -- to conduct proportionality analyses -- potentially personally identifiable information should be irrelevant. Indeed, Plaintiffs have previously represented that they do not desire to investigate users' viewing activities, and Viacom's general counsel is on record today stating that Viacom does not want to receive individuals' usernames and IP addresses. Accordingly, we request that Plantiffs agree that YouTube may redact usernames and IP addresses from the viewing data in the interests of protecting user privacy.

fan deaｔｈ

2008-07-03T14:14:00Z

I am falling in GOGATSUBYO". It means "May sickness". The economic year starts in April in Japan and many freshmen who join companies in April start to feel "something is wrong" or "this was not what I want" on May until they naturalize themselves to the new environment of their life.

Kim sent me this link. It's a funny movie. I guess this is a movie from the Ramens. The series of the shape of Japan.

this and this. But I missed this "fan death". Brian sent me about it yesterday. I didn't know that it's just a legend and I believed in it actually.

Listen, a few years ago, I used a fan all night long and the next day I found my forefinger was anesthetic. I thought I was almost dead because of the fan!! Actually I was amazed to find fan death in this urban legend topic, but I should try a fan for saving energy without any fear one night.

I write about urban legends sometimes like

Teriyaki Boyz - ZOCK ON! feat. Pharrell and Busta Rhymes

2008-07-03T07:11:36Z

From Cecily’s blog comes a heads-up notice of the Teriyaki Boyz’s new single ZOCK ON!, featuring Pharrell and Busta Rhymes. Produced by the Neptunes, I’m liking this track a truckload more than that ridiculous Fast and Furious track they did a while back. Also, Pharrell singing in Japanese and a video girl winking too much, if you’re into that sort of stuff.

YouAndYouAndYouTube: Viacom, Privacy and the Social Graph API

2008-07-03T13:27:58Z

From Wired via Thomas Roessler:

Google will have to turn over every record of every video watched by YouTube users, including users’ names and IP addresses, to Viacom, which is suing Google for allowing clips of its copyright videos to appear on YouTube, a judge ruled Wednesday.

I hope nobody thought their behaviour on youtube.com was a private matter between them and Google.

The Judge’s ruling (pdf) is interesting to read (ok, to skim). As the Wired article says,

The judge also turned Google’s own defense of its data retention policies — that IP addresses of computers aren’t personally revealing in and of themselves, against it to justify the log dump.

Here’s an excerpt. Note that there is also a claim that youtube account IDs aren’t personally identifying.

Defendants argue that the data should not be disclosed because of the users’ privacy concerns, saying that “Plaintiffs would likely be able to determine the viewing and video uploading habits of YouTube’s users based on the user’s login ID and the user’s IP address” .

But defendants cite no authority barring them from disclosing such information in civil discovery proceedings, and their privacy concerns are speculative. Defendants do not refute that the “login ID is an anonymous pseudonym that users create for themselves when they sign up with YouTube” which without more “cannot identify specific individuals”, and Google has elsewhere stated:

“We . . . are strong supporters of the idea that data protection laws should apply to any data that could identify you. The reality is though that in most cases, an IP address without additional information cannot.” — Google Software Engineer Alma Whitten, Are IP addresses personal?, GOOGLE PUBLIC POLICY BLOG (Feb. 22, 2008)

So forget the IP address part for now.

Since early this year, Google have been operating an experimental service called the Social Graph API. From their own introduction to the technology:

With so many websites to join, users must decide where to invest significant time in adding their same connections over and over. For developers, this means it is difficult to build successful web applications that hinge upon a critical mass of users for content and interaction. With the Social Graph API, developers can now utilize public connections their users have already created in other web services. It makes information about public connections between people easily available and useful.

Only public data. The API returns web addresses of public pages and publicly declared connections between them. The API cannot access non-public information, such as private profile pages or websites accessible to a limited group of friends.

Google’s Social Graph API makes easier something that was already possible: using XFN and FOAF markup from the public Web to associate more personal information with YouTube accounts. This makes information that was already public increasingly accessible to automated processing. If I choose to link to my YouTube profile with the XFN markup rel=’me’ from another of my profiles, those 8 characters are sufficient to bridge my allegedly anonymous YouTube ID with arbitrary other personal information. This is done in a machine-readable manner, one that Google has already demonstrated a planet-wide index for.

Here is the data returned by Google’s Social Graph API when asking for everything about my YouTube URL:

{
 “canonical_mapping”: {
  “http://youtube.com/user/modanbri”: “http://youtube.com/user/modanbri”
 },
 “nodes”: {
  “http://youtube.com/user/modanbri”: {
   “attributes”: {
    “url”: “http://youtube.com/user/modanbri”,
    “profile”: “http://youtube.com/user/modanbri”,
    “rss”: “http://youtube.com/rss/user/modanbri/videos.rss”
   },
   “claimed_nodes”: [
   ],
   “unverified_claiming_nodes”: [
    "http://friendfeed.com/danbri",
    "http://www.mybloglog.com/buzz/members/danbri"
   ],
   “nodes_referenced”: {
   },
   “nodes_referenced_by”: {
    “http://friendfeed.com/danbri”: {
     “types”: [
      "me"
     ]
    },
    “http://guttertec.swurl.com/friends”: {
     “types”: [
      "friend"
     ]
    },
    “http://www.mybloglog.com/buzz/members/danbri”: {
     “types”: [
      "me"
     ]
    }
   }
  }
 }
}

You can see here that the SGAPI, built on top of Google’s Web crawl of public pages, has picked out the connection to my FriendFeed (see FOAF file) and MyBlogLog (see FOAF file) accounts, both of whom export XFN and FOAF descriptions of my relationship to this YouTube account, linking it up with various other sites and profiles I’m publicly associated with.

YouTube users who have linked their YouTube account URLs from other social Web sites (something sites like FriendFeed and MyBlogLog actively encourage), are no longer anonymous on YouTube. This is their choice. It can give them a mechanism for sharing ‘favourited’ videos with a wide circle of friends, without those friends needing logins on YouTube or other Google services. This clearly has business value for YouTube and similar ’social video’ services, as well as for users and Social Web aggregators.

Given such a trend towards increased cross-site profile linkage, it is unfortunate to read that YouTube identifiers are being presented as essentially anonymous IDs: this is clearly not the case. If you know my YouTube ID ‘modanbri’ you can quite easily find out a lot more about me, and certainly enough to find out with strong probability my real world identity. As I say, this is my conscious choice as a YouTube user; had I wanted to be (more) anonymous, I would have behaved differently. To understand YouTube IDs as being anonymous accounts is to radically misunderstand the nature of the modern Web.

Although it wouldn’t protect against all analysis, I hope the user IDs are at least scrambled before being handed over to Viacom. This would make it harder for them to be used to look up other data via (amongst other things) Google’s own YouTube and Social Graph APIs.

Update: I should note also that the bridging of YouTube IDs with other profiles is one that is not solely under the control of the YouTube user. Friends, contacts, followers and fans on other sites can link to YouTube profiles freely; this can be enough to compromise an otherwise anonymous account. Increasingly, these links are machine-processable; a trend I’ve previously argued is (for better or worse) inevitable.

Furthermore, the hypertext and data environment around YouTube and the Social Web is rapidly evolving; the lookups and associations we’ll be able to make in 1-2 years will outstrip what is possible today. It only takes a single hyperlink to reveal the owner of a YouTube account name; many such links will be created in the months to come.

iPhone 2.0 OS rocks!

2008-07-03T14:28:00Z

Okay, okay, it’s early and I’ve only had it running for less than 24 hours but Apple made me very happy with the new 2.0 OS. I picked up an advance copy from a friend and have it running on my first gen iPhone hw.

I bought this first gen iphone on the first day it came out. And try as I might, it just became a very expensive ipod with a killer browser. I’ve been in hell - carrying around a blackberry for work and the iphone at the same time. Two devices everywhere just sucks.

But those days are gone. The new 2.0 OS addresses all the things I’ve been waiting for.

1. Full MSFT Exchange support. It just works. It took a little while to configure our server properly and open up the correct ports but once we did it my iphone was getting push email and wirelessly syncing with our company calender, phone book and mail. The only thing it doesn’t have is a task manager sync but I’m extremely happy with the performance.

2. Address book search. Up until now, the address book didn’t have a search function. That was a big problem for me. The new search box works well. Slower compared to the blackberry but it’s fine for me.

3. App store. I have a button now on my home screen that says App Store. It doesn’t work yet of course but I’m on the edge of my seat to download apps when they become available.

Other observations:

-the browser seems to be improved. text fields are easier to manage with the onscreen keyboard. Fonts get smaller as you type in more characters in the text field.

-still no copy & paste function :(

-promising location based services

-battery life with push email seems to be taking a hit or maybe it’s because I’m using my iphone constantly now :). I really wish there was a way to change batteries on the run.

-the keyboard still isn’t going to work for everyone. a lot of blackberry users are going to be tempted by the msft exchange support but won’t like the keyboard. It’s fine for me and I’m actually digging it

I can’t wait to get the iPhone 3G now. My blackberry is going to be sad & lonely in the desk drawer…

Update: one more thing. Attachments are amazing. I just opened up a 2 page excel spreadsheet and it looked great!

Twitter, worth a billion? nothing? too late? no, no & no.

2008-07-03T14:51:14Z

I try not to add too much to the twitter whinging (was that too british of me?), but I thought I’d give this a roll. It all pretty much confirms my generalized notion that there is no middle ground on the web.

The Beginning

Let’s start from the beginning, first there was the post that suggested how Twitter could be worth a billion in a year. I read that post in Reader and thought it was interesting, but ultimately, irrelevant. Twitter is going to be working on its scaling issues for the next several months at least - it isn’t doing anything new in a year. Ultimately it would be interesting to see a mobile payment system, but I’d guess they’d team up with Amazon or Google or PayPal to make that happen rather than do it themselves, it’d also remain to be seen if there was a strong desire to SMS people money and what the security implications of that would be.

The Backlash

However, the web elite took strong exception to this, because everyone knows that Twitter is over. Everyone’s moving to FriendFeed, or Plurk, or whatever. Twitter’s down messages on FriendFeed are replete with how over that service is and how FF is going to take over. Take a look at this article on CNet discussing how Twitter could be worth nothing in a year. Here he talks about how everyone’s moving to FriendFeed, how people are beginning to “see the light” and realizing how being down nearly 2% of the time (nearly 6 days out of the year!!) is too much to bear in the pursuit of letting people know what you’re eating for lunch. (ok, I kid, there’s more interesting stuff happening, but really…).

Then take a look at the author’s Twitter profile and his FriendFeed one. He’s got well over 1000 peoples on Twitter. On FriendFeed? 2. Granted you only see who he’s subscribed to, but seeing as how his ratio on Twitter is pretty close to 1:1, his overflowing inbox of FF subs means it either isn’t so overflowing or he doesn’t care about FF at all. Look, even Scoble, a major FF proponent and someone who my empirical observation is subscribed to by almost everyone on FF has only 3000 subscriptions, compared to close to 30,000 on twitter. And I’d bet you no shortage of money that the percentage of people who subscribe to him on FF is significantly greater than his share of Twitterers.

Another post from a really excellent blog (except for this particular post) says that Twitter has two more weeks before it reaches the point of no return. Two more weeks! In the world of people who use these services, no one, but no one knows about FriendFeed. Don’t get me wrong, I love FF, but really, ask your friends this. What’s email? What’s IM? What’s Facebook? What’s Twitter? What’s FriendFeed? What’s Plurk? Ask them that series of questions and see what happens. Right around Twitter you’ll notice an exponential fall off.

Twitter isn’t going anywhere

Only to the .1% of the online world who twitters a ton and have several hundreds if not thousands of followers does this downtime actually mean anything. For everyone else, if they even notice the downtime, they don’t think much of it. Just like how they live with their computers crashing all the time. It’s just one of those things that you deal with. And that .1% isn’t going to leave twitter if their thousands of followers don’t go with them. Look at the folks who say they are going to leave, at best they go to FriendFeed and then route all their followers through FF so they don’t miss anything. So what?

Twitter just brought in two big investors, not least of which is Bezos you might have heard of his website, Amazon.com? Yeah, turns out he probably knows some people who might know how one might scale up a website. I’m not sure about that, it’s just an inkling I’ve got. If I had cash and was awesome web guy, I wish I could have invested - this is the perfect time. The market has totally overshot the problem and Twitter’s got way more negative publicity than it warrants - Bezos probably got in for a steal and a year or two from now will rake in the profits. Or, you know, the sort of profits that Web2.0 sites don’t actually ever make till they do something like IPO or get bought.

The competitors

In the meantime, the competitors are lining up trying to grab as much share of angry first adopters as they can. Jaiku, Plurk and most newest comer Identi.ca. I joined a few of these just to see what all the hubbub was about.

Jaiku looks ok, but is in private beta and while it seems to be withering on the vine, murmurs of some magical google sauce can be heard through the leaves. Eh. (me on jaiku)

Plurk is, I believe, too annoying and contrived to do well with the slightly later adopters. It’s grown quickly, but that’s easy from a small base. I’d guess with its quirky interface and cute art it may get a niche, but won’t ever be that big. (me on plurk)

Identi.ca is the newest and in some ways most interesting competitor. It’s a twitter clone with two interesting features, one it’s free (yay GPLv3) and two it seems to be built with the idea that it will be a federated/decentralized service. So the load will be spread out everywhere and a server going down doesn’t affect everyone. It remains to be seen how well that works in real life and how resistant it is to failure and malicious intent. (me on identi.ca)

I’d guess these probably at best settle into some niche of users that sustains them. None of them come within even an order of magnitude of Twitter’s subscriber base in the foreseeable future. Twitter’s got an easy monopoly and it’ll take much worse than a 2% downtime to affect that. To sum up, Twitter’s not going to be worth a billion in a year. Twitter’s also not going to be gone in a year. Twitter will be doing just fine in a year and none of these competitors are going to be anywhere close to it. I’m just saying, where do you fall on this? Do you even use twitter and/or any competitors?

MySQL support fun, multiplication

2008-07-03T19:45:08Z

There was a question how to do an aggregate multiplication in MySQL. MySQL does not provide such functionality, so we were looking at various workarounds.

We discussed UDF interface that allows to construct custom aggregates, also did look at @a:=@a*field hack, and how different initializers have results wrapped differently.

Then Scott killed our discussion with this simple query:

select exp(sum(log(c)))

Of course, thats nice mathematical approach to solve the multiplication issue with just SUM() at hands, but while we were still in awe, Scott explained it with this wit:

The Great Flood is over, and as the animals are departing 3×3, Noah is blessing each, saying “Go forth, be fruitful and multiply.” Two snakes come down the ramp and say to Noah, “We can’t. We’re adders.”

Noah groans and says, “That’s the worst pun I’ve heard in 40 days and nights. Go sit in that pile of sticks until I can deal with you!”

After mucking out the ark, Noah returns to the sticks and lifts them up, to find baby snakes everywhere. “What happened?” he asks. “I thought you were having problems!”

The snakes reply, “Even adders can multiply with logs.”

:-)

Google, YouTube, Data

2008-07-03T13:53:36Z

While reading my RSS feeds this morning I picked up this:
YouTube Must Give All User Histories To Viacom

After Scientology's DMCA request on Slashdot we made an active choice to squash data on users to limit the possibility of this sort of request. We randomized incoming trackable data on users and tossed everything but aggregate data for long term storage.

Why?

For one, we simply did not need to keep terabytes of log data sitting around collecting dust. Secondly, while the data might be useful for determining trends we risked our user's privacy. We believed this was unacceptable. One court request and we could be handing over who knows what to any company that could find an uneducated judge to sign away the privacy of millions. The data was just not that valuable.

What would I like to see?

Sites handing control of data retention over to users.

It would be good to see more sites give users the opportunity to have their tracking information removed after a period of time from companies databases.

We could start a trend by having websites publish data retention policies .

So what would it take to make this happen? Would peer pressure work? I am not in favor of creating more laws.

What if we petition sites to make steps in this direction. A few at a time, with a goal of long term of putting peer pressure on sites that do not follow the lead of user privacy oriented sites.

Is this too much too ask for?

Google Talk for iPhone

2008-07-03T06:41:00Z

Google has finally launched a mobile version of Google Talk, but it's optimized for iPhone's browser. You can access if you go to google.com/talk on an iPhone/iPod Touch or using this permalink: http://talkgadget.google.com/talkgadget/m (shorter URL: http://tinyurl.com/4vnfcd).

This Ajax version is based on Google Talk's Flash gadget, but it doesn't include tabs, group chat, options to add contacts and send email. What you can do is to update your status, search your contacts and chat with people that are online.

Besides this mobile interface, Google also has a mobile app for Blackberry. That means if you don't have a Blackberry, an iPhone or at least a WebKit mobile browser, you have to find a third-party service.

eBuddy is a service that offers both a basic mobile interface for Google Talk and a mobile application that lets you chat faster with your friends. eBuddy doesn't connect only to Google Talk, it's a multi-network IM client that works with MSN, Yahoo and AIM. Unfortunately, I couldn't find any information regarding the way eBuddy stores or uses your credentials and seeing that eBuddy sends the password in plain text is worrisome.

Another mobile application that lets you chat with your Google Talk contacts is Fring, but it only works on Symbian and Windows Mobile devices. As usually, make sure you trust the service before entering the username and password of a Google account.

Google Talk for iPhone

2008-07-03T11:14:35Z

Google Talk, Google’s chat program, is now available for the iPhone, Google announced in one of their blogs. Google says it’s enough to just point your iPhone browser to www.google.com/talk and then sign in for this to work. Google says you’ll then be able to chat, view your contacts and search for them, and update your status message.

[Thanks Colin! Screenshot by Google, edited to remove minor noise.]

[By Philipp Lenssen | Origin: Google Talk for iPhone | Comments]

[Advertisement] Want to advertise here? Your ad will show in the blog and feed.

Apple to sell iPhones without AT&T contracts [Apple]

2008-07-01T17:40:00Z

US customers will be able to purchase new iPhones without locking themselves into a two-year contract with AT&T. It'll just cost an extra $400 — $599 for one with 8 gigabytes of storage, $699 for one with 16 gigabytes. Customers will still have to sign up for an AT&T wireless subscription, but it won't have the same penalties for changing carriers. Analysts figure it costs Apple about $173 to manufacture each iPhone, and believe Apple is selling the phones to AT&T at about $400 each. That means that at $599, Apple and AT&T are roughly splitting the extra $400 profit on an unlocked phone. Almost makes you wonder why AT&T bothers to sell subscriptions.

Engadget’s Ryan Block and Peter Rojas To Team On New Startup

2008-07-01T18:54:38Z

Engadget’s editor-in-chief Ryan Block will be leaving parent company AOL shortly, sources say, to launch a new startup. Partnering with him on the new company will be Peter Rojas, Engadget’s former editor-in-chief (pictured left below, next to Block).

Block officially became Engadget’s chief editor in June 2007 (although he actually took over the site in early 2006). He took the spot after founding editor in chief Rojas left the blog full time. Rojas went on to launch RCRD LBL in late 2007, an online record label that distributes free music.

Rojas will stay with RCRD LBL and will be working with Block part time on the new project, one source says.

The new startup will be a content site around technology, we’ve heard, but won’t be a blog. It may be a forum/social network based site to create lots of user generated content. What will it be called? We perused the 39 domain names that Block owns and one stood out because he also registered the .net and .org versions (all other domains he owns are .com): Devixe. Is that the new name of the startup? From what we’ve heard, it’s only one name that is being considered.

A lot of this is still speculation, and Block isn’t saying much other than he is still with AOL and has not resigned. As to whether he plans to in the near future, and whether he intends to launch Devixe or another site with Rojas, he won’t comment.

If he leaves, Block’s likely replacement as editor in chief is Josh Fruhlinger, the current managing editor and Block’s no. 2 at Engadget. Senior editor Thomas Ricker and senior associate editor Joshua Topolsky are other candidates, although Ricker lives in Amsterdam and Topolsky is also a music producer. Fruhlinger is the likely replacement.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Google silencing Obama critics? Memo to New York Times bloggers: ur doing it rong [Great Moments In Journalism]

2008-07-02T00:00:00Z

"Did Google use its network of online services to silence critics of Barack Obama?" asks New York Times reporter Miguel Helft today, in what reads like the Gray Lady's attempt to do Valleywag-style gossipmongering. There's something very wrong with the post: Read it and see if you think Helft believed for a minute that any Google employees deliberately and maliciously turned off a few Google-hosted blogs supporting Hillary Clinton and John McCain.

No, it reads like a classic IT malfunction. Second-tier bloggers were accidentally identified as splogs — spam blogs — and disabled. At worst, Google's computers were fooled by Obamatards who maliciously flagged other candidates' sites en masse as "objectionable," triggering an automated shutoff. That's a good enough story that it doesn't need to be wrapped in a far more serious pretend charge. Google silencing Obama critics? If Times editors thought for a moment it had really happened, the story wouldn't be on the Bits blog. It would be on Page 1.

Rickshaw Bagworks opens shop online

2008-07-02T12:58:30Z

The TED2008 Gift Bag was the first product from a brand-new company, Rickshaw Bagworks. Made in San Francisco with sustainable fabrics and thoughtful details, the TED bags became a bit of a cult item -- not least because they weren't available for retail sale at the time of the '08 conference.

This week Rickshaw opens its online store, selling the TED-style bag (they call it the "med commuter messenger") along with other styles, including a baby bag that benefits Healthy Child Healthy World.

Fox News: We Photoshop, You Decide

2008-07-02T18:35:00Z

There isn't much love lost between Rupert Murdoch's Fox News Channel and the New York Times. So, not surprising that Fox News morning hosts Steve Doocy and Brian Kilmeade didn't much like Times TV reporter Jacques Steinberg's story over the weekend on Fox News' ratings: "Fox News Finds Its Rivals Closing In."

So what did they do? Well, they had a little fun with Steinberg and TV editor Steve Reddicliffe, during "Fox and Friends" this morning. The poodle photoshop is clearly parody, but look how Steinberg and Reddicliffe's head shots come out on Fox's air -- yellowed teeth, exaggerated features, and in Steve's case, less hair.

Full segment from this morning's "Fox & Friends" below:

The Real Reason Powerset Sold (Out)

2008-07-02T22:00:18Z

When visiting Israel in the middle of summer, it’s generally not a good idea to go for a walk in the afternoon, even if it is along the sea. The heat and humidity sap your energy, making you feel as if you spent nearly three hours in the gym. But that wasn’t enough to stop me from writing a post about Microsoft buying Powerset for what is rumored to be around $100 million.

I’ve been unable to stop wondering why founder Barney Pell decided to take the money and run — after all, he used to turn blue in the face telling people how superior Powerset’s approach to search was. If it was so superior, Mike Masnick of Techdirt put it best when he wrote that “[T]he exit certainly falls well short of the hype around Powerset. If Powerset was actually seeing any traction at all it never would have agreed to sell at that price.”

To some extent, Mike is right, but I would add another reason: infrastructure, specifically how expensive it is to build. At our Hadoop meet-up earlier this year, Chad Walters, director of engineering at Powerset, noted that their search “requires 100 times more processing than simple keyword searching and indexing (about one second per sentence is required for processing).”

Powerset used some pretty nifty technologies to build out their system, but in order to really scale, they would have needed more money — a lot of it.

And Powerset would have had to scale; there’s no other way to compete with search’s 800-pound gorilla, Google. That’s why Microsoft is building a gigantic data center in the Chicago area focused almost entirely on search. (Which it can now use to help roll out Powerset’s search technology to a larger audience.)

This is an abject lesson for every startup looking to get into the business of search: No matter how good your algorithms are, you still have to deal with the cost of queries, which need to be low enough to be offset by some kind of advertising in order to make a profit. (The conspiracy theorist in me says that if your results are really good you won’t be able to generate enough inventory to serve up ads that bring in the dollars, but maybe I’m just too cynical.)

One of our readers believes that it is possible to build a search engine that surpasses Google’s. Nevertheless, as I’ve noted in the past, “[P]rocess-optimized infrastructure ensures that Google???s cost of executing a query keep going down” — and that allows the company to wring more dollars from the system.

Given all that, Powerset has done a good job of wringing a hundred million from Microsoft. Not that there’s anything wrong with that.

Bonus Link: Don Dodge of Microsoft explains the logic behind the deal.