California has a way of inventing things that turn out to be popular around the world (hey, not just Google). California passed the first so-called security breach notification law, in 2002. To date, 39 U.S. states have enacted laws that require notice if some form of personal information is compromised in a data security breach.

Since then, the trend has gone global. In August, the Office of the Privacy Commissioner of Canada issued guidelines on how to handle a security breach, which are just that – guidelines – but provide sensible recommendations for the handling of security breaches, including the notification to affected individuals where a breach creates a risk of harm. The logic behind the Canadian approach is that prompt notification to individuals in these cases can help them mitigate the damage by taking steps to protect themselves. New Zealand has followed a similar line by issuing guidelines on how to handle privacy breaches, which also focus on the role of notification to avoid or mitigate harm to individuals.

This trend is about to come to Europe too. The European data protection directives do not have any express provisions requiring companies that have suffered some sort of security breach to notify the individuals affected. The traditional thinking is that Europe does not need such a measure because there is already a well known obligation that calls for the adoption of appropriate technical and organisational measures to protect personal data against security breaches.

However, a European Commission consultation document of 2006 hinted at the prospect of security breaches notification obligations for providers of electronic communications networks and services, on the basis that network operators and ISPs, as the gatekeepers for users’ access to the online world, carry a special responsibility in this regard. This was followed by recommendations made by the Article 29 Working Party to extend those obligations to "data brokers," banks and other online service providers. The Working Party went on to say that for important breaches, all customers of the communications provider – not just those directly affected – should be informed.

The European Commission is now expected to include a formal proposal introducing mandatory security breach notifications or otherwise, into its review of the EU’s e-communications regulatory framework. Bearing in mind the experiences in other parts of the world and the latest thinking in jurisdictions like Canada and New Zealand, the risk of harm to the individual should be a determining factor in triggering notification obligations. Otherwise, the real risk is to trivialise notification obligations to such an extent that they become meaningless and ineffective in terms of data protection. In fact, the potential damage to consumers of a blanket notification obligation could be twofold: on the one hand, it can create unjustified anxieties and on the other hand, it may result in a lack of proper attention to more serious incidents.

A recent article about our treatment of a political ad placed on Google suggested that we make decisions about advertising content based on the political viewpoint of the advertiser and the ad. This is simply untrue. We do not accept or reject ads based on the political opinions expressed in the ads or the political views of the advertiser.

Let me explain the facts behind the matter and the policy behind the decision.

Recently, representatives of Senator Susan Collins' Senate re-election campaign tried to place an ad on Google that included a reference to MoveOn.org, a political group. The text of this ad was rejected by our system because of our trademark policy, not because of its political content.

Under our trademark policy, a registered trademark owner may request that its mark not be used in the text of other parties' ads. Some time ago, MoveOn.org submitted a request to Google that its trademark not be used in any ads, and as a result our advertiser support team offered instructions on how Senator Collins' campaign could edit and resubmit its ad.

Any company or organization -- regardless of political affiliation -- could do what MoveOn did and thereby prevent advertisers from running ads that include their trademarks in ad texts. And that's very important. The ad in question could have said that MoveOn.org was great, or even just so-so, and our policy would have resulted in the same outcome; Google would have asked the advertiser to drop the trademarked phrase.

Our trademark policy is considered one of the least restrictive in the industry. It strikes a balance among the interests of users, advertisers, and trademark owners by leaving it up to trademark owners to notify us of restrictions on their registered trademarks. Any entity that demonstrates to us that it owns trademark rights can request that its trademarked terms not be used in the text of Google ads.

We are committed to fairness and freedom of expression, and we recognize that the nature of political advertising is to inspire debate. We look forward to engaging in this debate in an open and transparent fashion, and we encourage political candidates and campaigns to contact our elections team with any questions they may have about our policies.

"Our acquisition of DoubleClick does not foreclose other companies from competing in the online advertising space. Rather, the transaction is just one of several that underscore the strong competition in the online advertising space...Each of the acquisitions following our purchase of DoubleClick demonstrates that there are many sophisticated, well-financed, and competitive companies that believe that the online advertising space merits more investment and remains open to strong competition."

"We make privacy a priority because our business depends on it. If our users are uncomfortable with how we manage the information they provide to us, they are only one click away from switching to a competitor’s services. If you don't believe me, recall that before Google, users clicked on an earlier generation of search engines like Excite, Altavista, Lycos, and Infoseek – each extremely popular in its time. User interests effectively regulate our behavior, and user trust is a critical component of our business model."

I've blogged before about Google's view that our acquisition of DoubleClick will lead to better, more useful online advertising for consumers, and more choices for advertisers and website publishers. We have also seen that the acquisition has been followed by significant investment, innovation, and competition in the online advertising world. Just yesterday, for example, Microsoft announced that it had added 20 new advertising clients after closing its acquisition of aQuantive, a DoubleClick competitor. We see this as yet more evidence that companies are competing in the online advertising space and the free market.

With the the Antitrust Subcommittee of the Senate Judiciary Committee taking a look at online advertising later this week, we thought you'd appreciate seeing what several independent folks have said about competition in the online advertising space and about the DoubleClick acquisition:

Los Angeles Times - Editorial
"The market for advertising online is still in its infancy… And neither Google, DoubleClick nor anyone else dominate the emerging market for video advertising, which in the broadband era may emerge as the most effective and lucrative sector yet." (Los Angeles Times, Editorial 4/17/2007)

Financial Times - Editorial
“Google and DoubleClick are different kinds of business. Buying DoubleClick does not increase Google's share of the total web audience, a more meaningful measure of the market.” (Financial Times, Editorial, 5/25/07)

James B. Stewart, Columnist, Wall Street Journal
"For starters, Google and DoubleClick aren't direct competitors. Google specializes in online search and accompanying text ads. DoubleClick specializes in so-called web display advertising, especially video. It's hard to see how the combined companies would dampen competition when they don't compete to begin with." (Wall Street Journal, 4/18/2007)

Thomas Eisenmann, Harvard Business School
“If the merger is approved ‘there ought to be room for multiple players’ in the online advertising space…’Natural monopolies are very rare, and this is not one of them.’” (Technology Daily PM, 7/18/07)

Andrew Frank, Analyst, Gartner
"'It seems there is a clear distinction between Google's business and the business it is entering with the acquisition of DoubleClick,'..If so, Google is not acquiring extra market power through the proposed deal." (Financial Times, 4/16/2007)

John Deighton, Professor, Harvard Business School
"Google has had a long history of resisting the impulse to exploit individual surfing histories. I think they understand that it's even more important to show that restraint now." (Los Angeles Times, 4/17/2007)

Gene Munster, Senior Research Analyst, Piper Jaffray
“Fears that advertisers are worried about sharing info [between Google and DoubleClick] are likely overblown.” (Note, 4/23/07)

• The rapidly changing business landscape of online advertising, and the role it plays in providing free, accessible, user-friendly, and high-quality content to consumers. Since 2000, annual online revenue in the U.S. alone grew from $8 billion to over $17 billion. The growth in online advertising has also spurred innovation, competition, and investment in the online advertising space – all of which produce consumer benefits in the form of more online resources and more relevant information. In our experience consumers value the advertisements that we deliver along with search results and other web content, which connect them to the information, products, and services they seek. Simply put, advertising is information, and relevant advertising is information that is useful to consumers.
  
  
• The ways in which online advertising is contributing to a healthy and vibrant small business community. We know that many website owners can afford to dedicate themselves to their sites more fully – and sometimes full-time – because a significant percentage of the revenue we earn from advertising ends up in their hands as publishers of blogs and other websites and our advertising partners. In 2006, Google paid $3.3 billion in revenue to our partners. Our advertising network helps small businesses connect in an affordable and effective manner with otherwise unreachable consumers, including consumers in small, remote, or niche markets. An advertiser decides exactly how much money to spend on advertising, and can tie its spending directly to the response of a potential customer.

Of course, we continue to focus our attention on the privacy of our users, and privacy ought to be an important component of the Town Hall. We recognize that user, advertiser, and website publisher trust is critical to the success of our business, and we've taken a number of recent steps to help bolster our already strong privacy policies. We also think our acquisition of DoubleClick provides an opportunity for us to bolster privacy even further.

We're looking forward to talking more about these issues at the Town Hall.

• Damages apportionment. Damages should be calculated based on the fair share of the patent’s contribution to the value of a product, and not on the value of a whole product that has many components. So for example, a windshield wiper found to an infringe a patent should not spur a damage award based on the value of the entire car.
• Restricting forum-shopping. Certain district courts have become notorious for rarely invalidating a patent, and have tilted the balance too often in favor of plaintiffs. We support judicial venue provisions to ensure that patent lawsuits are brought only in district courts with a reasonable connection to the case.
• Post-grant review. The patent system should include a meaningful second chance for the U.S. Patent and Trademark Office to review potentially problematic patents in a timely way, thereby promoting high-quality patents.
  
• Willfullness. Patent infringers can be forced to pay triple the damages in cases where they are found to have "willfully" infringed a patent, but that standard has been devalued. Punitive triple damages should be reserved for cases of truly egregious conduct.

“Freedom” is a word that gets used a lot in Washington, but what does it mean, exactly, for Google and its users? Tuesday night, our CEO Eric Schmidt told the Progress & Freedom Foundation’s annual Aspen Summit that freedom and openness are the core principles that helped make possible the Internet's – and Google's -- birth and growth.

The Internet was built on open standards, Eric noted, and that those platforms are really platforms of economic opportunity and free expression. Free markets and open standards have led to so much innovation, that it’s usually best for government not to rush into regulating new technology. And he cited both political empowerment (such as the YouTube presidential debates) and economic empowerment (like the $3 billion that we paid out to website owners last year through our advertising partnerships) as the fruits of Internet freedom.

In the policy arena, Eric offered three specific calls to action. First, he said we need to defend freedom of speech as more speech comes online – and give teeth to the issue by pressing governments to classify censorship as a trade barrier. Second, we need to continue working toward universal broadband access, by government collaborating with industry and making sure that networks remain content neutral. And third, he called on government to be more transparent to its citizens – citing as an example our Sitemaps partnership with the federal government and five state governments.

But freedom wasn’t the only thing on Eric’s mind Tuesday. The PFF crowd was particularly interested in Google’s positions on both the upcoming 700 MHz spectrum auction and on net neutrality (and posed a few questions skeptical of our stance). Without announcing any definitive plans to bid, and cautioning that we're still carefully evaluating our options, Eric indicated that Google “probably” would decide to participate in the auction.

Check out the complete video of Eric’s talk, and tell us what freedom on the 'Net means to you.

Those who complain about Google's purchase of DoubleClick make two claims. Both are flawed.

The first argument is that, since both firms have a large market share of their respective spheres, a merger would be monopolistic. The flaw is that the two companies undertake activities that don't overlap. Google places text ads mainly on its own Web sites and search-result screens. DoubleClick delivers display ads from advertisers to Web sites. It creates no ads and controls no Web sites. Even if we believe that Internet advertising is a distinct market (debatable, since it comprises only about 5% of all advertising) the combined firms will not gain any market power since they do not have any business in common.

The second argument comes from privacy advocates who have filed a brief with the FTC. They say the merger "could impact the privacy interests of 233 million Internet users in North America." The FTC's antitrust function and its consumer protection function are fundamentally different. Indeed, the more information markets have, the more competitive they are. If "privacy" advocates have their way, there would be less information and markets would not work as well.

Coveted bits of the radio spectrum called "white spaces" -- unused areas of spectrum wedged between licensed TV channels -- may soon be freed up by the Federal Communications Commission. Right now no broadband devices are allowed to use these parts of the spectrum, but the FCC is considering whether to let companies sell FCC-certified wireless devices that would be used without an exclusive broadcast license in these slivers of bandwidth. Such white-space devices (WSDs) would be low-power and so would emit signals over very small geographic areas. White space within the TV band is unlicensed, like WiFi, but is physically better suited than WiFi for broadband transmission.

Google and other companies (including Dell, EarthLink, Hewlett-Packard, Intel, Microsoft, and Philips) have formed the "White Spaces Coalition," to persuade the FCC to establish appropriate interference standards that would allow entrepreneurs to develop fixed and mobile devices that utilize these airwaves. Earlier this year, the coalition submitted two prototype devices (from Microsoft and Philips) to the FCC's engineers to demonstrate the feasibility of this approach.

The FCC's engineering analysis, released two weeks ago, confirms what we have stated all along: it is technologically feasible to provide Internet access through this segment of spectrum without interfering with either digital television signals or wireless microphones. While one of the prototypes unfortunately was damaged, the other prototype fully demonstrated the promise of using these "white spaces" for Internet access. The coalition filed comments at the FCC yesterday responding to these test results.

FCC Chairman Kevin Martin has expressed a keen interest in keeping this matter moving forward, and the coalition will be working with FCC staff to address any remaining technical issues. As the Washington Post notes today, the promise that this spectrum holds for bringing the Internet to more Americans is too great to ignore:

Just two months ago, the notion that the FCC would take such a big step forward to give consumers meaningful choice through this auction seemed unlikely at best. Today -- thanks in no small part to broad public support for greater competition -- the FCC has embraced important principles of openness, and endorsed the unfettered workings of the free market for software applications and communications devices. Moreover, over the last few weeks several leading wireless carriers have reversed course and for the first time acknowledged our call for more open platforms in wireless networks. By any measure, that's real progress.

By the same token, it would have a more complete victory for consumers had the FCC adopted all four of the license conditions that we advocated, in order to pave the way for the real "third pipe" broadband competition that FCC Chairman Kevin Martin has been touting. For our part, we will need time to carefully study the actual text of the FCC's rules, due out in a few weeks, before we can make any definitive decisions about our possible participation in the auction.

In the meantime, we thank Chairman Martin for his leadership, and his compelling insight that American consumers deserve better in the wireless and broadband worlds. We've also had the pleasure of working on this issue with a broad cross-section of public interest groups that understand the need to foster more choices and competition in the wireless and broadband worlds.

The upcoming FCC 700 MHz spectrum auction certainly has spurred both lively debate and, at times, heated rhetoric. With the FCC set to vote tomorrow on the rules for the auction, I thought it would be useful to summarize briefly where things stand at this point.

FCC Chairman Kevin Martin has stated from the beginning that his number one priority is to make broadband available to all Americans through a “third pipe” to the home (in addition to telephone and cable company broadband service). In support of that viewpoint, the Chairman has taken a bold stand for consumer choice by proposing that licensees must allow the use of any device or application on a specified portion of the 700 MHz spectrum. This approach -- if crafted with appropriately effective and enforceable provisions -- would free consumers from burdensome and artificial constraints on what they can do with their phones and software. These license conditions for the first time will enable device and applications competition at the "edges" of the wireless network.

Unfortunately, these same conditions fall well short of Chairman Martin's own goal of fostering the creation of a third pipe competitor. As long as incumbents are motivated by a desire to protect their current business models, and can continue to use a “blocking premium” to thwart fair market rates, they have every incentive to outbid would-be rivals. Such an auction outcome will constitute business as usual -- with no new broadband options in sight for consumers.

Google has joined numerous public interest groups and other Web companies in seeking more fundamental “wholesale open access” conditions. We believe these additional conditions would ensure that, no matter who wins the auction, consumers, along with service providers of all shapes and sizes, will have a seat at the table. We even committed to invest at least $4.6 billion in such a scenario, despite the fact that we have not traditionally been a communications company. Some have criticized us for, in their view, rigging the auction to our own benefit. We think quite the reverse is true: only by imposing certain openness conditions will potential new market entrants have a fair shot at successfully bidding in the auction.

Openness, user choice, and innovation have been elements fundamental to the rise and success of the Internet. We believe those same elements are critical for even the possibility of new broadband competition in the wireless space. If the FCC ultimately decides not to adopt "wholesale open access" license conditions, we do not see how significant new competition can emerge from this auction.

The time for debate is drawing to a close. The five FCC commissioners are this moment contemplating the relative merits of the parties' arguments, and are set to make a final decision on Tuesday morning. The prospects for fostering robust competition in this slender but valuable slice of spectrum hangs in the balance.

• On April 30, Yahoo announced its intent to acquire Right Media, an online advertising exchange, for $680 million.

• On May 16, AOL announced its plans to acquire ADTECH AG, an online ad-serving company, for an undisclosed amount.

• On May 17, WPP Group announced its planned acquisition of online advertising company 24/7 Real Media for $649 million.

• On May 18, Microsoft announced its planned acquisition of aQuantive, an online advertising firm, for $6 billion.

And just this week, two more announcements highlighted the tremendous activity in this space:

• On July 24, AOL announced its acquisition of TACODA, an online behavioral targeting advertising network, for an undisclosed amount.

• Finally, yesterday Microsoft announced that it has agreed to acquire online advertising exchange AdECN Inc. for an undisclosed amount.

What does all this mean? It means that each of the leading Internet companies believe that they can position themselves to succeed in the online advertising space -- through the free market, and without government intervention. These companies believe that there are many ways to compete in this business.

Google, Microsoft, AOL, Yahoo, and others are developing different combinations of capabilities in an effort to provide the most compelling offering to advertisers, publishers, and customers. For example, Microsoft’s purchase of aQuantive will eventually result in it owning an ad serving business that competes with DoubleClick, and will also make Microsoft one of the largest interactive advertising agencies in the U.S. In DoubleClick, Google is acquiring a technology that delivers and measures the performance of display ads – a technology that is critical if we are to compete in display advertising.

Beyond the different approaches that companies are taking, more capital infusion into the online ad business also means that more entrepreneurs will enter it, too. In fact, we have noticed that several startups in the online advertising space have received venture funding since April. More entrepreneurs, more market participants, and more capital are combining to create more competition and innovation.

Brian McAndrews, the President and CEO of aQuantive (which, as noted above, has been purchased by Microsoft) recently said about online advertising: "We're in the first or second inning of a long game here. There's no monopoly on innovation. I don't think you're going to see two or three big players and then game over. There will continue to be a broad range of companies." We couldn’t agree more.

In recent days, and especially following Eric Schmidt's July 20 letter to FCC Chairman Kevin Martin, many people have asked us a straightforward question: why don't you just attempt to win the spectrum bidding outright, and then implement an open wholesaling business model yourself? Or, as AT&T has put it, "put up or shut up."

That question makes a lot of sense, especially for those most familiar with the ordinary marketplace structures they see on eBay, or Home Shopping Network, or at Target. In those everyday cases, the buyers and sellers collectively determine what is the fair market value for something, based on what willing participants on both sides agree should be the price. The free market is the optimal market.

But an FCC spectrum auction is a very different animal. Unlike in most commercial transactions, participants in an FCC auction operate in an artificially defined market. Initially, the spectrum comes sliced and diced in predetermined packages of varying bandwidth, geography, and duration. Those discrete slices tend to be compatible with what regulators perceive to be the prevailing services and technologies of the moment, such as centralized voice communications.

Further, given the sizable investments involved, only well-capitalized corporations can afford to bid at auctions. Ordinary citizens or entrepreneurs with novel ideas don't even show up. Even so, players still come to the table with unique assets, and in some cases disparate business models. For the upcoming 700 MHz auction in particular, the issue boils down to the different incentives at work between the existing national wireless carriers -- the incumbents -- and those companies seeking to enter the market for the first time -- potential new entrants.

As we have seriously considered entering the 700 MHz auction, we have been consulting with auction experts and game theorists to help us better understand the dynamics of a typical spectrum auction. What they have been telling us is that in a head-to-head bidding war between an incumbent wireless carrier and a potential new entrant, the incumbent almost invariably will prevail. Why? The answer involves two key economic factors: what we call the "incumbent blocking premium" and the "incumbent dilution discount."

Incumbent blocking premium

A significant economic factor that comes into play in an auction environment is the incentive and ability of one of the players to thwart the designs of the other. In the context of an FCC spectrum auction, there are at least three pertinent elements.

First, incumbent wireless carriers come to the auction with a vast array of existing assets, including thousands of radio towers, tens of thousands of miles of communications "backhaul" networks, and millions of customers, along with numerous retail outlets and tons of advertising. And perhaps most important of all, incumbents already own lots of spectrum -- much of which the FCC gave away for free some years ago, rather than sold at auction. By contrast, a true new entrant has none of these assets. Thus, to an incumbent, purchasing another wireless license is just an incremental investment, one made that much less costly given the existing, readily-available business inputs. To a new entrant, facing the daunting challenges of actually building and operating a network for the first time, the investment is less certain.

Second, the incumbent has the added benefit of operating in a less than fully competitive environment. Some use the term "monopoly rents" to describe the situation where a company enjoys revenues and profits that exceed what normally would be the case in a robustly competitive environment. Potential new entrants do not enjoy the same advantage.

Third, and perhaps most important, the incumbents have every incentive to preserve and protect their existing business model. Given their investment in all the necessary business inputs, and the relatively high prices and low bandwidth characteristics of their existing service offerings, the incumbents must prevent the entry of potential competitors to the market. In a spectrum auction, this means paying whatever it takes to block new entry. Not surprisingly, economists call this a "blocking premium."

As the diagram below shows, these three elements together mean that an incumbent will almost always be in a position to outbid a potential new entrant, simply by bidding above and beyond the fair market price. Unlike in a healthy auction situation, the final price would be higher than otherwise would be commercially reasonable.

The second significant economic factor that comes into play in any auction environment is related to the number of players who actually show up to participate in the bidding. Again, in a normal commercial environment, market prices are shaped by the number of willing buyers, from just a few to potentially millions. However, an FCC spectrum auction presents a comparatively artificial scenario.

Given the existence of the incumbent blocking premium, as described above, it is often the case that there are no new potential entrants to bid against an incumbent. Why should a new player even bother to bid, or bid aggressively, if the incumbent inevitably will prevail? Where there may be only two incumbents -- or even one -- left to bid for a license, obviously the resulting price will not reflect the fair market value that otherwise would have been reached. The dilution of competitive bidders means the final price will be lower than otherwise would be the case. Recent studies have confirmed that this is a pervasive aspect of the FCC auction environment.

Un-skewing the spectrum auction

When looking at the combined impact of the incumbent blocking premium and the incumbent dilution discount, it's easy to see how FCC auction results become skewed.

Ironically enough, it is Google that has been accused of attempting to skew the auction structure, by our recommendation that the licenses be conditioned on certain "open platforms" requirements. Of course, as we have explained it is the current auction system that skews the results away from potential new entrants and in favor of existing incumbents.

Our position is simple enough. FCC Chairman Kevin Martin and the other commissioners have argued persuasively that we need a real third pipe broadband competitor in this country. They also believe that the upcoming 700 MHz auction is the best way to get there. All we are saying is that, based on what we know, new broadband competition will emerge from the upcoming auction only if the FCC's rules allow it to happen. For Google, and other potential new entrants, the prevailing imbalance can be corrected most effectively by introducing license conditions based on open platforms.

While Google embraces the kinds of openness and innovation that are the hallmark of the Internet, the incumbents apparently prefer their existing business models. That of course is their prerogative. However, open platforms -- specifically, open applications, open devices, open wholesale services, and open network access -- together make the spectrum more valuable to Google, or any other potential bidder seeking to create innovative, higher-speed, lower-priced offerings.

That is why Google has indicated that it is willing to spend a minimum of $4.6 billion in the auction, which is the FCC's reserve price for the particular spectrum block in question. At the same time, incumbents are unable to leverage anti-competitive blocking in this scenario. Regardless of who wins the bidding, however, the end result is an auction that yields a fair market price, with the added bonus of a new broadband network that is open to all comers. The American people get full value for their spectrum, plus open broadband platforms -- and even the possibility of a real third pipe competitor. Not a bad deal overall.

• Open applications: consumers should be able to download and utilize any software applications, content, or services they desire;
  
• Open devices: consumers should be able to utilize their handheld communications device with whatever wireless network they prefer;
  
• Open services: third parties (resellers) should be able to acquire wireless services from a 700 MHz licensee on a wholesale basis, based on reasonably nondiscriminatory commercial terms; and
  
• Open networks: third parties (like Internet service providers) should be able to interconnect at any technically feasible point in a 700 MHz licensee's wireless network.
  

Citizens should have a right to privacy online. And governments have an obligation to keep their citizens safe. Finding the right balance between privacy and security is a delicate balancing act. Europe’s recent experience with data retention holds interesting lessons for everyone concerned with this balance.

In the aftermath of the Madrid bombings in 2004, the European Council adopted a Declaration on Combating Terrorism, which stated the need for rules on the retention of communications traffic data by European service providers for the first time. In some European countries, the ability to monitor communications was perceived as a practical priority in helping law enforcement agencies prevent and investigate terrorist acts. In April of 2004, the UK, Sweden, Ireland and France put forward a proposal for a Framework Decision calling for the retention of a wide variety of data for between 12 and 36 months.

However, for some politicians, the idea of adopting wide-ranging measures, requiring providers of telecommunications and Internet services to retain details of calls and electronic communications for periods of time beyond their pure operational needs, was not entirely justified. Indeed, for a while European privacy rights appeared to have the upper hand and the European Union institutions seemed to listen to the objections of the European Parliament’s Committee on Civil Liberties, Justice and Home Affairs.

According to the calculations of this group of European Members of Parliament, if all the traffic data covered by the proposal did indeed have to be stored, the network of a large Internet provider would accumulate up to 40,000 terabytes – the equivalent of four million kilometers worth of paper files -- or about 10 stacks of files each reaching from Earth to the moon. But others pointed out that even the slowest terrorist would figure out that he could simply avoid his communications being traced by using a non-European service provider. Nonetheless, the political pressure continued, and the European Commission went on to propose a directive on data retention in September 2005.

The rest is history… and now law. Although the European Parliament’s Committee on Civil Liberties, Justice and Home Affairs succeeded at introducing some amendments aimed at softening the effect of the proposal, an unprecedented data retention directive was adopted by the European Council on 15 March 2006. This directive imposes retention obligations between six months and two years in relation to accessible data generated or processed as a consequence of a communication or a communication service.

On paper, the aim behind the directive is simple and proper: to harmonise data retention rules across the EU and to ensure that the necessary information is available for the purpose of the investigation, detection and prosecution of serious crime. Unfortunately, the simplicity pretty much ends there. For a start, using the words “directive” and “harmonisation” in the same sentence is often an oxymoron, especially when a directive is cobbled together as a compromise between conflicting ideological positions.

On a practical level, the likelihood of seeing a consistent implementation of the rules across the EU is effectively zero. The timing of the implementation – due by September 15, 2007 – will certainly vary. 16 of the 27 EU Member States have already declared that they will delay the implementation of data retention of Internet traffic data for an additional period of 18 months, as permitted by the directive. The compulsory retention period for each type of data will also vary from country to county (e.g. Germany has proposed 6 months, the UK 12 months, and the Netherlands 18 months). The interpretation of other key elements, such as “serious crime," “competent national authorities,” or “electronic communications services” will be different across jurisdictions too.

These uncertainties impact on the justification for any privacy intrusions. Is a country more democratic than its neighbour because of its shorter retention period? Or do the citizens of that country face a greater security risk for the same reason? If there is something about the data retention directive that can be called into question is its proportionality – not necessarily in terms of financial cost to service providers, but in terms of privacy and anonymity loss. And what will Internet companies do in practice, especially if they operate one data architecture that cannot vary from one country to another: apply the longest retention period, or the shortest, or some “average”?

The data retention directive is of course just part of the picture. Several other initiatives provide additional evidence of the fact that traditional concepts of Internet privacy are in turmoil. One example was a proposal by the German government to complement its anti-terrorism measures by prohibiting the use of anonymous email accounts, by mandating that service providers verify the identity of their account holders.

Thankfully, the German government has recently retracted this proposal. Nonetheless, the idea continues to appeal to many: to make sure that every single e-mail user can be tracked down to an identifiable individual, so that the police can locate the terrorist behind the e-mail with the bomb-making instructions attachment, to take the most blatant possible example. The issue once again is whether this threat to anonymity on the Internet will be effective in making the world a safer place. Or will it do nothing to catch your average technology-savvy terrorist while eroding yet another layer of Internet privacy?

So, against this background, what is Google doing? We have recently announced a new policy to anonymize our search server logs after 18 months (we’re the first in our industry to have taken this step). We’re trying to get the balance right too, between privacy and other goals (like security, fraud prevention, and search improvements). People want to be free as much as they want to be safe. That’s true online too.

As I’ve written before, Google has become increasingly involved in U.S. spectrum policy issues this year. One of our top public policy objectives is to expand the Internet's reach to more Americans. In part, that means creating new competition to challenge the existing broadband access duopoly (between cable and phone companies), by paving the way for consumers to gain meaningful alternatives via advanced wireless services.

Unfortunately, the wireless airwaves required to develop such a service traditionally have been allocated in a fragmented and inefficient manner. The federal government’s upcoming auction of spectrum in the 700 MHz bands (as part of the digital television transition) offers a tremendous, and probably unique, opportunity to promote competition and web-based innovation.

Earlier this year, Google and other members of the “Coalition for 4G in America” urged the Federal Communications Commission to adopt flexible rules that encourage competitive entry by new and innovative broadband companies. At the time, we stated that our advocacy in the 700 MHz proceeding did not necessarily signal our intention to participate in the auction itself, although no final decision had yet been made.

In comments we filed in late May, we stressed that new entrants face considerable hurdles when competing head-to-head with incumbent wireless carriers. We also noted that a proposal by Frontline Wireless to impose a wholesale/open access mandate on a certain spectrum block would ensure that the owner of that block at least would operate its wireless network in an open manner.

Over the last several weeks, we’ve been taking a closer look at whether and how Google might participate meaningfully in the auction. As part of that look, we've consulted with spectrum auction experts and conducted various game theory scenarios. Our analysis has confirmed that, under the originally proposed rules, the existing national wireless carriers are likely to prevail in the bidding process against a potential new entrant like Google. While we remain interested in the possibility of participating in the auction, it’s clear that the incumbent carriers have built-in advantages that will prove difficult to overcome (particularly the economic and operational barriers to entry for a company like ours, and the relatively greater value and usefulness that spectrum brings to existing carriers).

What would happen if one or some of the existing national wireless carriers win this valuable spectrum at auction? They would probably use it to protect their existing business models and thwart the entry of new competitors -- both understandable actions from a rational business perspective. Beyond the loss of a valuable public resource, however, that outcome would not bring us any closer to fostering much-needed competition in the broadband market, or providing innovative new web applications and service offerings.

Too much is at stake for the federal government to let that happen. Late yesterday, we filed a letter urging the FCC to take concrete steps to make sure that regardless of who wins the spectrum at auction, consumers’ interests are best served. We believe that the winning bidders should be required to adhere to enforceable rules that require the adoption of four types of "open" platforms:

• Open applications: consumers should be able to download and utilize any software applications, content, or services they desire;
• Open devices: consumers should be able to utilize a handheld communications device with whatever wireless network they prefer;
• Open services: third parties (resellers) should be able to acquire wireless services from a 700 MHz licensee on a wholesale basis, based on reasonably nondiscriminatory commercial terms; and
• Open networks: third parties (like internet service providers) should be able to interconnect at a technically feasible point in a 700 MHz licensee's wireless network.

We believe that adopting these four license conditions collectively will encourage prospective broadband companies to participate in the auction, and be able to bid successfully for the available spectrum. Not only are new entrants more likely to embrace an ethos of openness, but additional forms of competition will emerge from web-based entities, such as software applications providers, content providers, handset makers, and ISPs. And consumers ultimately will come out ahead in that rich and vibrant broadband environment.

In the meantime, there is now potentially positive news coming out of the FCC. Chairman Kevin Martin apparently is about to circulate proposed auction rules to his fellow commissioners, and we're hearing through the proverbial grapevine that his proposal includes several of the open platform conditions we have recommended. If these reports are accurate, we are most encouraged by this favorable development. Obviously we'll need to see the fine print, but such a proposal would represent a step forward for new, innovative entrants to the broadband market.

I've addressed some of your comments about the broadband market and differentiating web traffic based on type. Now I'll turn to another big subject of your comments on net neutrality.

Doug, Keith, Drywall, asokoloski, psmith, and others engaged in a lively debate about whether and how Google and other Web companies compensate the telecom infrastructure providers for our use of their network facilities. As many well know, the Internet’s longstanding charging arrangements allow each party to pay for its own connection to the Internet. That party then is free to utilize that connection in whatever lawful ways are desired. Google believes that consumers should be able to acquire higher speed or performance capacity from the broadband providers, and then use this capability to reach any service they wish on the Internet. In particular, consumers should be able to purchase tiered pricing arrangements, based on the use of bandwidth, latency requirements, or other objective measures. Such arrangements would constitute an appropriate, cost-based practice that fully compensates the broadband provider for the additional capabilities provided.

On the other end of the “pipe,” Internet-based companies spend billions of dollars annually on R&D to create and deploy compelling content, applications, and services for American consumers. This massive amount of material typically is deployed on millions of Web servers located around the country. In order for the content and applications to be delivered into the Internet, so it then can be made available to consumers, Web companies must arrange with network operators to: carry the data traffic from company facilities to their Web servers over local telecom lines (the “last mile”); carry the data traffic from the Web servers into the Internet over high-speed, high-capacity data lines (“special access”); and carry the data traffic over the numerous interconnected networks that make up the Internet (the “Internet backbone”). To accomplish these important connectivity and transport functions in a fast and effective manner, Internet companies collectively pay many billions of dollars per year to network operators, which fully compensates them for their network investment.

We believe that broadband providers should be precluded from charging content providers for terminating traffic to a particular end user. Allowing broadband providers to leverage their “situational monopoly” over terminating traffic would allow them to choose which content providers receive preferential treatment over others, thereby distorting the marketplace. The institution of terminating charges also could lead to the balkanization of the Internet, in which each of the hundreds of local telephone and cable operators around the country – and, perhaps even more importantly, around the world -- would assess its own set of fees for terminating traffic on its network.

I hope these clarifications have been helpful, and that you'll keep sharing your thoughts.

P.S.: Be sure to check out Robert Cannon’s outstanding blog, Cybertelecom, which should be required reading for anyone interested in the Internet and broadband policymaking discussions in D.C.

Yesterday I addressed some of the comments on my net neutrality post dealing with the broadband market. Today I'll delve a little deeper on another issue you asked about: type-based traffic differentiation.

Several users commented on Google’s position that reasonable type-based differentiation of Internet traffic can be an acceptable business practice. As we explained in our FCC comments, we do not dispute that broadband providers should have the ability to manage their networks, as well as engage in a broad array of business practices. To us, the real question comes down to what kinds of business models and network management techniques rely on unilateral control over last-mile broadband facilities (the proverbial “on-ramps” to the Internet), in the service of anticompetitive or discriminatory intent.

Most known network management techniques will create few if any marketplace harms. So, for example, we believe that a broadband provider should have the leeway to utilize legitimate application and content-neutral network management practices that seek to neutralize objective network harms. These practices would include halting harmful denial of service (DOS) attacks, or blocking certain traffic containing viruses or worms.

We also stated that it may be a reasonable business practice to prioritize all packets of a certain application type. Our rationale for that position is that there may well be tangible end user benefits from giving preferential treatment to certain Internet packets, such as those in a streaming video transmission, in order to enhance the end user experience. As long as the categories of “type” are identified and designed with objective criteria in mind (such as sensitivity to latency or jitter), and prioritization is apply in an even-handed manner to all packets in that category, the practice can be a fair one. If, on the other hand, type-based prioritization is used to promulgate discriminatory practices – such as degrading or prioritizing certain applications based on an intention to impair the offerings of competitors – such practices should be prohibited as unreasonable.

I will be the first to say that allowing type-based prioritization is a close call, and reasonable minds certainly can differ. Many in the Internet community lack trust that the broadband provider will employ packet prioritization over last-mile networks in a manner that still preserves an open Internet environment and does not facilitate the introduction of anticompetitive practices. Moreover, prioritization generally creates a host of practical, economic, and technical problems, not least of which is that the broadband carrier has fewer incentives to build out its network capacity where it can make more money simply by charging for differentiated service.

On balance, though, we believe that the possible end user benefits from differentiating between certain broad categories of Internet traffic outweigh the potential competitive and discriminatory threat. That doesn’t mean that we cannot subsequently criticize, and seek to halt, any such practices that take an anticompetitive turn. Nor does it mean that Google somehow is going “soft” on network neutrality. We have merely drawn the line in a slightly different place than others in the pro-net neutrality camp.

Tomorrow, I'll address your comments about another net neutrality topic: paying for bandwidth.

Thanks to all who read my initial posting on network neutrality, and especially to those folks who took the time to leave comments. While I don’t have the personal bandwidth (ouch) to respond to each and every posting while also taking care of my “day job” here at Google, I will check back periodically and offer follow-up reactions.

I believe it is important for companies like Google to establish a place of meaningful dialogue with the general public, and to open our policy advocacy role to outside analysis -- and yes, criticism. I also welcome your thoughts on other telecommunications and media policy issues of interest to you (my own current favorite topic is the FCC’s ongoing consideration of rules governing the upcoming 700 MHz auction). And I urge folks to take their views to the places where they ultimately count: the well-trod halls of the FCC and the U.S. Congress.

Today, I'll offer some thoughts on one of the key issues raised in some of the comments on my net neutrality post: the broadband market. Later this week I'll address two other issues you asked questions about: type-based traffic differentiation, and payment for bandwidth.

Market analysis

Scott Cleland asked whether the search market is as highly concentrated as the broadband market, and thus deserves network neutrality regulation as well. Scott asked me the same question at an EDUCAUSE policy conference last month, but I’m happy to repeat my response and elaborate here.

I’m certainly no economist, but I do try to keep up on the latest thinking about how markets function. The available evidence demonstrates that the U.S. consumer broadband market is highly concentrated, with extensive barriers to entry, high consumer switching costs, and no near-term competition. By stark contrast, the search market is robustly competitive, with numerous major players, new near-term competition, no significant barriers to entry, and zero user switching costs.

• First, the broadband market suffers from a pronounced and intractable lack of competition. At best, consumers have a choice today between a telephone company and a cable company. The Congressional Research Service has described the current market as a “broadband duopoly,” where telephone and cable companies face little real competition. The FCC’s own skewed July 2006 figures still showed an overwhelmingly concentrated broadband market, with telephone companies and cable companies controlling access to 99.6 percent of all U.S. consumers. The share of alternative broadband platforms also has been decreasing steadily over time, from a less-than-impressive 2.9 percent in 1999 to an anemic 0.4 percent today. The GAO further found that only about 28 percent of all US households subscribed to broadband service in 2005, and noted that DSL and cable modem service together constitute the only broadband technologies actually available to consumers.
  
  By comparison, the market for search engines in the United States is highly competitive. Stats from comScore and other market analysis firms show that Google has only about half of the overall U.S. search market. Indeed, Google competes every day with large, well-funded companies like Yahoo, Microsoft, AOL, and Ask.com. Aggregator search services such as dogpile.com also flourish, along with dozens of other popular search-based services in the U.S. alone. In short, the U.S search market is anything but concentrated.

• Second, while emerging technologies may eventually enable viable competitors, such channels currently do not compete in terms of speed, price, availability, or technological maturity. In fact, each of the supposed technology alternatives –- such as broadband over powerline (BPL), satellite internet, and 3G wireless -- provide no real competitive option. In particular, 3G wireless fails the test because, among other drawbacks: (1) most services do not qualify as “high speed” under the FCC’s current definitions; (2) data plan prices typically are at least double what consumers pay for cable or DSL service; (3) wireless providers block many common Internet applications and services, foreclose outside network attachments, and reserve the right to terminate service arbitrarily for using “non-conforming” services; (4) few consumers have substituted wireless broadband service for wireline broadband service; and (5) the FCC’s figures include all owners of 3G phones, whether or not they have purchased or used them for Internet access. Perhaps most significantly, the largest national wireless high speed Internet providers represent two incumbents from the wireline market and two longstanding telecommunications provider. The appropriate way to add up the available consumer options is not by simply counting individual broadband technology platforms, but rather independent platforms.
  
  By contrast, the search market is dynamic and expanding all the time. Not only do we seen a raft of new entrants in the text-based search market, but also nascent services such as video search, image search, news search, and other specialized search functions. No company can afford to rest on its laurels in this ongoing race for faster and better search functionality.

• Third, considerable and insurmountable barriers to entry also limit the possibility of new competition. To build and operate a nationwide broadband system capable of competing head-on with the incumbents, would-be market entrants must (among other things) pour tens of billions of dollars into constructing local, regional, and national communications infrastructure, pay for backhaul, access rights of way, and interconnect with hundreds of other U.S. carriers. On top of that enormous investment, the market entrant then must create a commercially viable service offering, complete with retail sales outlets, technical and customer support, and advertising.
  
  By contrast, barriers to entry in the search market are quite low. Even though established search engines from Yahoo, Infoseek, MSN, Altavista, and many others had a considerable head start in the late 1990s, Google showed how a good idea hatched on a neutral and open Internet can change the industry in a few short years. Of course, any individual or company with an algorithm, and a means of accessing the Internet, can pave their own way into the burgeoning search engine market.

• Fourth and finally, even assuming the ability to choose another broadband provider in a particular area, consumers endure considerable switching costs. Providers typically bind their customers with multi-year contracts (sometimes termed “stickiness”), bolstered by substantial early termination penalties. The prevalence of bundling together different services also helps providers reduce “churn,” where there are competing offerings. Equipment costs, truck rolls, and even legacy email accounts all create disincentives for consumers to move to another broadband service provider.
  
  By contrast, it is the user of search engines that possesses all the power. If an end user decides he or she no longer likes a preferred search engine, the time and cost to change search engines is zero. Changing search engine preferences – as with many other Web-based businesses -- is literally just a mouse click away. As a result, stickiness is not a common feature of Web-based entities.

Together, these salient factors -- excessive market concentration, no viable competitors, considerable consumer switching costs, and substantial barriers to entry -- should lead policymakers to conclude that there is a major competition problem in the broadband market. No such problems exist in the search market.

I'll have more to say later this week about some of the other issues you've raised. In the meantime, what do you think?

• The information industries –- broadly understood to mean Internet companies, book and periodical publishers, broadcasters, and the music and film industries –- together comprise a critical and growing component of the U.S. economy. They create jobs, spur economic growth, and bring to the world the best of American ideals about freedom of expression, creativity, and innovation.
  

• To industries that depend upon free flows of information to deliver their services across borders, censorship is a fundamental barrier to trade. For Google, it is fair to say that censorship constitutes the single greatest trade barrier we currently face.
  
  
• Some forms of censorship are entirely justifiable: the worldwide prohibitions on child pornography and copyright infringement, for example. Others, however, are overbroad and unwarranted. When a government blocks the entire YouTube service due to a handful of user-generated videos that violate local sensibilities –- despite our willingness to IP-block illegal videos from that country –- it affects us as a non-tariff trade barrier.
  
  
• Just as the U.S. government has, in decades past, utilized its trade negotiation powers to advance the interests of other U.S. industries, we would like to see the federal government take to heart the interests of the information industries and treat the elimination of unwarranted censorship as a central objective of our bilateral and multilateral trade agendas in the years to come.
  

Local and state officials can layer and tailor secure information about their jurisdictions and feed it into a broader database that will give state and federal decision makers valuable and timely information. With existing state GIS (Geographic Information System) and orthophotographic data, we are able to transform massive amounts of useful information into a common operational picture. Examples of real-time applications include emergency evacuation routing, vehicle and asset tracking, critical infrastructure mapping, plume modeling, real-time sensor feeds, real-time streaming video, risk visualization, and post-event imagery placed alongside pre-event imagery.

• Prioritizing all applications of a certain general type, such as streaming video;
• Managing their networks to, for example, block certain traffic based on IP address in order to prevent harmful denial of service (DOS) attacks, viruses or worms;
  
• Employing certain upgrades, such as the use of local caching or private network backbone links;
• Providing managed IP services and proprietary content (like IPTV); and
  
• Charging consumers extra to receive higher speed or performance capacity broadband service.
  

• Levying surcharges on content providers that are not their retail customers;
• Prioritizing data packet delivery based on the ownership or affiliation (the who) of the content, or the source or destination (the what) of the content; or
  
• Building a new "fast lane" online that consigns Internet content and applications to a relatively slow, bandwidth-starved portion of the broadband connection.