Google Data » Panayiotis Mavrommatis

Safe Browsing Alerts for Network Administrators is graduating from Labs

Panayiotis Mavrommatis — Thu, 06 Oct 2011 16:54:00 +0000

Posted by Nav Jagpal, Security Team

Today, we’re congratulating Safe Browsing Alerts for Network Administrators on its graduation from Labs to its new home at http://www.google.com/safebrowsing/alerts/

We announced the tool about a year ago and have received a lot of positive feedback. Network administrators, large and small, are using the information we provide about malware and phishing URLs to clean up their networks and help webmasters make their sites safer. Earlier this year, AusCert recognized our efforts by awarding Safe Browsing Alerts for Network Administrators the title of “Best Security Initiative.”

If you’re a network administrator and haven’t yet registered your AS, you can do so here.

Protecting users from malicious downloads

Panayiotis Mavrommatis — Tue, 05 Apr 2011 18:27:00 +0000

Posted by Moheeb Abu Rajab, Google Security Team

For the past five years Google has been offering protection to users against websites that attempt to distribute malware via drive-by downloads — that is, infections that harm users’ computers when they simply visit a vulnerable site. The data produced by our systems and published via the Safe Browsing API is used by Google search and browsers such as Google Chrome, Firefox, and Safari to warn users who may attempt to visit these dangerous webpages.

Safe Browsing has done a lot of good for the web, yet the Internet remains rife with deceptive and harmful content. It’s easy to find sites hosting free downloads that promise one thing but actually behave quite differently. These downloads may even perform actions without the user’s consent, such as displaying spam ads, performing click fraud, or stealing other users’ passwords. Such sites usually don’t attempt to exploit vulnerabilities on the user’s computer system. Instead, they use social engineering to entice users to download and run the malicious content.

Today we’re pleased to announce a new feature that aims to protect users against these kinds of downloads, starting with malicious Windows executables. The new feature will be integrated with Google Chrome and will display a warning if a user attempts to download a suspected malicious executable file:

Download warning

This warning will be displayed for any download URL that matches the latest list of malicious websites published by the Safe Browsing API. The new feature follows the same privacy policy currently in use by the Safe Browsing feature. For example, this feature does not enable Google to determine the URLs you are visiting.

We’re starting with a small-scale experimental phase for a subset of our users who subscribe to the Chrome development release channel, and we hope to make this feature available to all users in the next stable release of Google Chrome. We hope that the feature will improve our users’ online experience and help make the Internet a safer place.

For webmasters, you can continue to use the same interface provided by Google Webmaster Tools to learn about malware issues with your sites. These tools include binaries that have been identified by this new feature, and the same review process will apply.

Phishing URLs and XML Notifications

Panayiotis Mavrommatis — Thu, 14 Oct 2010 19:42:00 +0000

Posted by Nav Jagpal, Security team

Recently, we announced Safe Browsing Alerts for Network Administrators. Today we’re adding phishing URLs to the notification messages. This means that in addition to being alerted to compromised URLs found on networks, you’ll be alerted to phishing URLs as well.

We’d also like to point out the XML notification feature. By default, we send notification messages in a simple email message. However, we realize that some of you may want to process these notifications by a script, so we’ve added the ability to receive messages in XML format. Click on an AS in your list to modify preferences, such as enabling the XML notification feature. If you decide to use XML email messages, you should familiarize yourself with the XML Schema.

If you’re a network administrator and haven’t yet registered your AS, you can do so here.

Phishing URLs and XML Notifications

Panayiotis Mavrommatis — Thu, 14 Oct 2010 19:42:00 +0000

Safe Browsing Alerts for Network Administrators

Panayiotis Mavrommatis — Tue, 28 Sep 2010 20:30:00 +0000

Posted by Nav Jagpal and Ke Wang, Security Team

Google has been working hard to protect its users from malicious web pages, and also to help webmasters keep their websites clean. When we find malicious content on websites, we attempt to notify their webmasters via email about the bad URLs. There is even a Webmaster Tools feature that helps webmasters identify specific malicious content that has been surreptitiously added to their sites, so that they can clean up their site and help prevent it from being compromised in the future.

Today, we’re happy to announce Google Safe Browsing Alerts for Network Administrators -- an experimental tool which allows Autonomous System (AS) owners to receive early notifications for malicious content found on their networks. A single network or ISP can host hundreds or thousands of different websites. Although network administrators may not be responsible for running the websites themselves, they have an interest in the quality of the content being hosted on their networks. We’re hoping that with this additional level of information, administrators can help make the Internet safer by working with webmasters to remove malicious content and fix security vulnerabilities.

To get started, visit safebrowsingalerts.googlelabs.com.

Safe Browsing Alerts for Network Administrators

Panayiotis Mavrommatis — Tue, 28 Sep 2010 20:30:00 +0000

Meet skipfish, our automated web security scanner

Panayiotis Mavrommatis — Fri, 19 Mar 2010 17:49:00 +0000

Posted by Michal Zalewski

The safety of the Internet is of paramount importance to Google, and helping web developers build secure, reliable web applications is an important part of the equation. To advance this goal, we have released projects such as ratproxy, a passive security assessment tool; and Browser Security Handbook, a comprehensive guide for web developers. We also worked with the community to improve the security of third-party browsers.

Today, we are happy to announce the availability of skipfish - our free, open source, fully automated, active web application security reconnaissance tool. We think this project is interesting for a few reasons:

High speed: written in pure C, with highly optimized HTTP handling and a minimal CPU footprint, the tool easily achieves 2000 requests per second with responsive targets.
Ease of use: the tool features heuristics to support a variety of quirky web frameworks and mixed-technology sites, with automatic learning capabilities, on-the-fly wordlist creation, and form autocompletion.
Cutting-edge security logic: we incorporated high quality, low false positive, differential security checks capable of spotting a range of subtle flaws, including blind injection vectors.

As with ratproxy, we feel that skipfish will be a valuable contribution to the information security community, making security assessments significantly more accessible and easier to execute.

To download the scanner, please visit this page; detailed project documentation is available here.

Do machines dream of electric malware?

Panayiotis Mavrommatis — Thu, 29 Oct 2009 22:17:00 +0000

Posted by: Oliver Fisher, Anti-Malware Team

We've explored Google's anti-malware processes several times recently, as well as our efforts to work with webmasters to help protect their users. However, there's been some confusion about the objectivity of our scanning and flagging procedures.

Google uses fully automated systems to scan the Internet for potentially dangerous sites. These systems help detect sites infected with malware and then add a warning that appears in Google search results and in many web browsers. We flag sites in this way to help protect users who might visit them. The warning is a cautionary page, and we never prevent users from viewing the affected site if they choose. It's important to note that sites are often compromised without the webmaster's knowledge, so we provide affected webmasters with further information on the issues we've identified — including showing snippets of the malicious code we find. We also offer free resources in Google Webmaster Tools to help site owners clean their sites and request a re-scan.

Site owners sometimes say that we've made a mistake and that their site does not contain malware. For example, the recent appearance of a malware warning on people.com.cn sparked discussion about how Google flags websites. Our scanners — which are automated and indifferent to a site's subject matter — first found a malicious ad on the book.people.com.cn domain at approximately 3:47 a.m. PT on October 17, 2009. Over several days, the scanners detected thousands of URLs with suspicious content in other people.com.cn domains.

Malicious content can be very difficult to detect. A previous post on this blog offered tips for finding hidden malware and cleaning up websites. There are also good tips on Google's Webmaster Central Blog. If a webmaster has indeed removed the malicious content and filed a malware review request in Webmaster Tools, the warning label will be removed shortly. If it persists, however, it's very likely that dangerous content remains. Our scanners are highly accurate, and false positives are extremely rare.

When Google's automated systems detect dangerous content on a site, an email is sent to several administrative email addresses at the site, as well as to the corresponding Webmaster Tools account if one exists. We sent a notification to people.com.cn at 11:01 a.m. PT on October 17, just as any compromised site would receive. The email includes an explanation of how the site may have become compromised and unknowingly been distributing malware. It also describes the process of removing malware from the site and getting the Google warning removed from the site. A copy of the message sent to the addresses associated with infected sites is below:

We recently discovered that some of your pages can cause users to be infected with malicious software. We have begun showing a warning page to users who visit these pages by clicking a search result on Google.com.
...
We strongly encourage you to investigate this immediately to protect your visitors. Although some sites intentionally distribute malicious software, in many cases the webmaster is unaware because:
1) the site was compromised
2) the site doesn't monitor for malicious user-contributed content
3) the site displays content from an ad network that has a malicious advertiser

If your site was compromised, it's important to not only remove the malicious (and usually hidden) content from your pages, but to also identify and fix the vulnerability. We suggest contacting your hosting provider if you are unsure of how to proceed. StopBadware also has a resource page for securing compromised sites: http://www.stopbadware.org/home/security Once you've secured your site, you can request that the warning be removed by visiting http://www.google.com/support/webmasters/bin/answer.py?answer=45432 and requesting a review. If your site is no longer harmful to users, we will remove the warning.

As the email says, the fastest way for a site to be removed from the malware list is for the webmaster to file a review request via Google Webmaster Tools. Google's automated scanners will periodically re-examine the site even if no such request is received, but the process will take longer. People.com.cn did not file a review request, but our scanners reviewed the site on October 23 and removed the malware warning after finding that the malicious ad was gone.

Malicious display ads are an increasingly common way for sites to unknowingly distribute malware. We recently wrote about the steps that Google takes to help protect our advertising networks. Also, other publishers have recently written about their experiences with deceptive display ads.

Best Practices for Verifying and Cleaning up a Compromised Site

Panayiotis Mavrommatis — Thu, 22 Oct 2009 17:38:00 +0000

Written by Panayiotis Mavrommatis, Security Team

As part of Cyber Security Awareness Month, Google's Anti-Malware Team is publishing a series of educational blog posts inspired by questions we've received from users. October is a great time to brush up on cyber security tips and ensure you're taking the necessary steps to protect your computer, website, and personal information. For general cyber security tips, check out our online security educational series or visit http://www.staysafeonline.org/. To learn more about malware detection and site cleanup, visit the Webmaster Tools Help Center and Forum.

In our last post in this series, we explained Google's malware scanning process and how malware warning reviews work. It's not always clear to webmasters how to go about cleaning up their sites once they've been compromised, so this time we thought we'd share some best practices.

1) Verify Your Site with Google Webmaster Tools

If you have added and verified your site's ownership with Google Webmaster Tools, you can view a partial list of URLs where our system has detected suspicious content on your site, as well as samples of the malicious code. Once you've thoroughly cleaned up your site and addressed the vulnerability that allowed it to be compromised, it's easy to request a review through Webmaster Tools. We recognize that some site owners may want to use these tools even if they haven't already signed up with Webmaster Tools. For that reason, we enable you to verify ownership of your sites at any time, even if our systems have listed them as potentially dangerous.

2) If Your Site Has Been Compromised, Perform a Comprehensive Cleanup

If any part of your site has been compromised, thoroughly check all pages on the site for harmful code or content — not just the example pages listed in Webmaster Tools. Be sure to identify and address the underlying vulnerability that led to the compromise, or else reinfection is likely to occur.

Remember to Check Your Web Server Configuration

In addition to checking the contents of your site's pages and web server source code, remember to check that your web server configuration has not been modified by any intruders. If your web server has been compromised, your site's error pages can be modified to include custom HTML that actually redirects visitors to malicious sites.

Deleted & Error Pages: Dark Corners of Your Website Where Malware May Be Lurking

When a page is deleted from a site, the web server returns an error code (usually 404: Not Found) when requests to the "deleted" URLs are made. In addition to the error code in the HTTP header, the web server may send a custom error page or "Not Found" page, usually intended to help users find what they are looking for. If your site is infected, its error page can contain arbitrary HTML that exposes your visitors to malware. You can search our Webmaster Forum for information about how others are dealing with similar problems. The recently-launched malware samples feature in Google Webmaster Tools could also come in handy.

3) If You Switch Hosting Providers, Disable Access to the Old Version of Your Site

When a site is moved to a different hosting provider, the DNS records are updated such that the domain name points to a new IP address. In some cases, DNS caching can cause your domain name to continue resolving to the old IP address for some visitors even after the site has moved. For this reason, we recommend instructing your former hosting provider to stop serving any content for your site. This may cause some visitors to experience server errors for a few hours, but can protect them from visiting a potentially dangerous web server.

As always, our Webmaster Forum and StopBadware's BadwareBusters can be good sources of help and information when cleaning up a compromised site.

Show Me the Malware!

Panayiotis Mavrommatis — Mon, 12 Oct 2009 22:53:00 +0000

written by Lucas Ballard, on behalf of the Anti-Malware, Anti-Malvertising, and Webmaster Tools teams

As part of Cyber Security Awareness Month, we're highlighting cyber security tips and features to help ensure you're taking the necessary steps to protect your computer, website, and personal information. For general cyber security tips, check out our online security educational series or visit http://www.staysafeonline.org/. To learn more about malware detection and site cleanup, visit the Webmaster Tools Help Center and Forum.

To help protect users against malware threats, Google has built automated scanners that detect malware on websites we've indexed. Pages that are identified as dangerous by these scanners are accompanied by warnings in Google search results, and browsers such as Google Chrome, Firefox, and Safari also use our data to show similar warnings to people attempting to visit suspicious sites.

While it is important to protect users, we also know that most of these sites are not intentionally distributing malware. We understand the frustration of webmasters whose sites have been compromised without their knowledge and who discover that their site has been flagged. We proactively offer help to these webmasters: we send email to site administrators when we encounter suspicious content, we provide a list of infected pages in Webmaster Tools, and we maintain a service that allows webmasters to notify us when they have cleaned their sites. Read more about this process in the previous post on this blog.

We're happy to announce that we've launched a feature that enables Google to provide even more detailed help to webmasters. Webmaster Tools now provides webmasters with samples of the malicious code that Google's automated scanners detected on their sites. These samples — which typically take the form of injected HTML tags, JavaScript, or embedded Flash files — are available in the "Malware details" Labs feature in Webmaster Tools. Registered webmasters (registration is free) of infected sites do not need to specially enable the feature — they will find links to it on the Webmaster Tools dashboard. Webmasters will see a list of their pages that we found to be involved in malware distribution and samples of the malicious content that Google's scanners encountered on each infected page. In certain situations we can identify the underlying cause of the malicious code, and we'll provide these details when possible. We hope that the additional information will assist webmasters and help prevent their visitors from being exposed to malware.

Malware details for your site

Malware details for a particular page

While we're excited to offer this feature, we caution webmasters to use the tool only as a starting point in their site clean-up process. Google's scanners may not be able to provide malware samples in all cases, and the malware samples may not be a complete list of all the malware on the page. More importantly, we advise against simply removing the examples that are displayed in Webmaster Tools. If the underlying vulnerability is not identified and patched, it is likely that the site will be compromised again.

In addition to helping the webmasters of sites with malware warnings, this new detail is also designed to promote the general health of the web. In some cases, our automatic scanners find questionable content on a site but do not have enough data to add it to the malware list. The new "Malware details" feature will highlight these instances to webmasters early on to help them identify and address security vulnerabilities more quickly.

We hope you never have cause to use this feature, but if you do, it should help you quickly purge malware from your site and help protect its visitors. We plan to improve our algorithms in the upcoming months to provide even greater coverage, more accurate vulnerability identification, and faster delivery to webmasters.

The Malware Warning Review Process

Panayiotis Mavrommatis — Fri, 09 Oct 2009 17:38:00 +0000

written by Lucas Ballard and Ke Wang, Anti-Malware Team

As part of Cyber Security Awareness Month, Google's Anti-Malware Team is publishing a series of educational blog posts inspired by questions we've received from users. October is a great time to brush up on cyber security tips and ensure you're taking the necessary steps to protect your computer, website, and personal information. For general cyber security tips, check out our online security educational series or visit http://www.staysafeonline.org/. To learn more about malware detection and site cleanup, visit the Webmaster Tools Help Center and Forum.

Google's anti-malware efforts are designed to be helpful to both webmasters and website visitors. Google continuously scans our web index for pages that could be dangerous to site visitors. When we find such pages, we flag them as harmful in our search results, and also provide this data to several browsers so that users of these browsers will receive warnings directly. We undertake this process as part of our security philosophy: we believe that if we all work together to identify threats and stamp them out, we can make the web a safer place for everyone. While we believe these processes are important steps in helping to protect our users, we also understand the frustration felt by the webmasters of flagged sites. This is why we notify webmasters as soon as we discover that their sites have been compromised. Additionally, we provide webmasters with a tool to file a review once they have cleaned their site. The review process works as follows.

Part 1: The webmaster's job: The first step is site cleanup. The webmaster should remove all harmful content from the site. We realize that it can be tricky to find all the infections on a website, and webmasters should look thoroughly if the warning label persists. Keep in mind that if your site contains elements from another website that may have been compromised, it will remain flagged. This is because your site could still introduce harm to visitors. To prevent reinfection, the webmaster should also identify and fix the underlying software vulnerability that led to site compromise in the first place. For a guide on how to do this, visit stopbadware.org/home/security.

Once a webmaster has cleaned up the site, a Malware Review can be filed with Google's Webmaster Tools (please note that a Malware Review request is not the same as an Index Reinclusion request). The process for Malware Review is as follows:

Log in to Webmaster Tools.
From the Tool's home page click on the link to the site that is being flagged. This will bring you to the site's Dashboard.
There should be a large red banner across the top of the dashboard that says "This site may be distributing malware." Clicking on the link that says "More Details" expands the dashboard to reveal a list of pages on the site that were found to be malicious.
Below this list is a link that says "Request a review." A webmaster can fill out this form and click the "Request a review" button to initiate the review process.

More detailed instructions can be found here.

Part 2: Our job: Upon receiving a Malware Review request, an automated set of algorithms verifies that the site has been cleaned. These algorithms revisit a subset of both the malicious and non-malicious pages that were scanned when the site was originally flagged. Additionally, these algorithms test some pages that were not originally scanned. If none of the tested pages are found to be malicious, the site is deemed to be safe, and warnings are removed from search results. A typical appeal takes only several hours to complete, although in some cases the process may take up to one day.

In addition to processing appeal requests from webmasters, we also rescan compromised sites periodically.

We encourage webmasters of infected sites to quickly clean their web pages and proactively request reviews through Webmaster Tools. After the site has been thoroughly cleaned and reviewed, it will no longer show a warning on Google's search results pages or through the browsers making use of our data.

Ask the Google Anti-Malware Team

Panayiotis Mavrommatis — Fri, 14 Aug 2009 19:10:00 +0000

Posted by Fabrice Jaubert

The Google Anti-Malware engineering team knows you have many questions related to our scanning and flagging of infected sites, some with short and simple answers and some with more complex answers. The short-answer questions are already -- we hope -- adequately handled on the Webmaster Forums; now we want to do a better job at answering the more complex questions.

To this end, we have created a Google Moderator page for you to submit your questions, and to vote on other webmasters' questions. In two weeks (on Friday the 28th of August), we will close the page and select a few of the top-rated questions. Over the course of the next several weeks, we will do our best to answer each of these in a write-up, to be published here and to the Webmaster Malware Forum.

We hope to repeat this exercise (with a fresh Moderator page) in the fall to give you the opportunity to ask more questions.

Thank you, and see you on the Moderator page!

Reducing XSS by way of Automatic Context-Aware Escaping in Template Systems

Panayiotis Mavrommatis — Tue, 31 Mar 2009 21:06:00 +0000

Posted by Jad S. Boutros, Security Team

Building on our earlier posts on defenses against web application flaws ["Automating Web Application Security Testing", "Meet ratproxy, our passive web security assessment tool"], we introduce Automatic Context-Aware Escaping (Auto-Escape for short), a functionality we added to two Google-developed general purpose template systems to better protect against Cross-Site Scripting (XSS).

We developed Auto-Escape specifically for general purpose template systems; that is, template systems that are for the most part unaware of the structure and programming language of the content on which they operate. These template systems typically provide minimal support for web applications, possibly limited to basic escaping functions that a developer can invoke to help escape unsafe content being returned in web responses. Our observation has been that web applications of substantial size and complexity using these template systems have an increased risk of introducing XSS flaws. To see why this is the case, consider the simplified template below in which double curly brackets {{ and }} enclose placeholders (variables) that are replaced with run-time content, presumed unsafe.



  
    Hello {{USERNAME}}, view your Account.

In this template, four variables are used (not in this order):

USER_NAME is inserted into regular HTML text and hence can be escaped safely by HTML-escape.
USER_ACCOUNT_URL is inserted into an HTML attribute that expects a URL and therefore in addition to HTML-escape, also requires validation that the URL scheme is safe. By allowing only a safe white-list of schemes, we can prevent (say) javascript: pseudo-URLs, which HTML-escape alone does not prevent.
USER_COLOR is inserted into a Cascading Style Sheets (CSS) context and therefore requires an escaping that also prevents scripting and other dangerous constructs in CSS such as those possible in expression() or url(). For more information on concerns with harmful content in CSS, refer to the CSS section of the Browser Security Handbook.
USER_ID is inserted into a Javascript variable that expects a number as it is not enclosed in quotes. As such, it requires an escaping that coerces it to a number (which a typical Javascript-escape function does not do), otherwise it can lead to arbitrary javascript execution. More variants may be developed to coerce content to other data types, including arrays and objects.

Each of these variable insertions requires a different escaping method or risks introducing XSS. To keep the example small, we excluded several contexts of interest, particularly style tags, HTML attributes that expect Javascript (such as onmouseover), and considerations of whether attribute values are enclosed within quotes or not (which also affects escaping).

Auto-Escape

The example above demonstrates the importance of understanding the precise context in which variables are being inserted and the need for escaping functions that are both safe and correct for each. For larger and complex web applications, we notice two related vectors for XSS:

A developer forgetting to apply escaping to a given variable.
A developer applying the wrong escaping for that variable for the context in which it is being inserted.

Considering the sheer number of templates in large web applications and the number of untrusted content they may operate on, the process of proper escaping becomes complicated and error prone. It is also difficult to efficiently audit from a security testing perspective. We developed Auto-Escape to take that complexity away from the developer and into the template system and therefore reduce the risks of XSS that would have ensued.

A Look at Implementation

Auto-Escape is a functionality designed to make the Template System web application context-aware and therefore able to apply automatically and properly the escaping required. This is achieved in three parts:

We determined all the different contexts in which untrusted content may be returned and provided proper escaping functions for each. This is part science and part practical. For example, we did not find the need to support variable insertion inside an HTML tag name itself (as opposed to HTML attributes) so we did not build support for it. Other factors come into play, including availability of existing escaping functions and backwards compatibility. As a result, part of that work is template system dependent.

We developed our own parser to parse HTML and Javascript templates. It provides methods which can be queried at a point of interest to obtain the context information necessary for proper escaping. The parser is designed with performance in mind, and it runs in a stream mode without look-ahead. It aims for simplicity while understanding that browsers may be more lenient than specifications, particularly in certain corner cases.

We added an extra step into the parsing that the template system already performs to locate variables, among other needs. This extra step activates our HTML/Javascript parser, queries it for the context of each variable then applies its escaping rules to compute the proper escaping functions to use for each variable. Depending on the template system, this step may be performed only the first time a template is used or for each web response in which case some limitations may be lifted.

A simple mechanism is provided for the developer to indicate that some variables are safe and should not be escaped. This is used for variables that are either escaped through other means in source code or contain trusted markup that should be emitted intact.

Current Status

Auto-Escape has been released with the C++ Google Ctemplate for a while now and it continues to develop there. You can read more about it in the Guide to using Auto-Escape. We also implemented Auto-Escape for the ClearSilver template system and expect it to be released in the near future. Lastly, we are in the process of integrating it into other template systems developed at Google for Java and Python and are interested in working with a few other open source template systems that may benefit from this logic. Our HTML/Javascript parser is already available with the Google Ctemplate distribution and is expected to be released as a stand-alone open source project very soon.

Co-developers: Filipe Almeida and Mugdha Bendre

OAuth for Secure Mashups

Panayiotis Mavrommatis — Wed, 19 Nov 2008 01:41:00 +0000

Posted by Eric Sachs, Senior Product Manager, Google Security

A year ago, a number of large and small websites announced a new open standard called OAuth. This standard is designed to provide a secure and privacy-preserving technique for enabling specific private data on one site to be accessed by another site. One popular reason for that type of cross-site access is data portability in areas such as personal health records (such as Google Health or Microsoft Healthvault), as well as social networks (such as OpenSocial enabled sites). I originally became involved in this space in the summer of 2005, when Google started developing a feature called AuthSub, which was one of the pre-cursors of OAuth. That was a proprietary protocol, but one that has been used by hundreds of websites to provide add-on services to Google Account users by getting permission from users to access data in their Google Accounts. In fact, that was the key feature that a few of us used to start the Google Health portability effort back when it was only a prototype project with a few dedicated Googlers.

However, with the development of a common Internet standard in OAuth, we see much greater potential for data portability and secure mash-ups. Today we announced that the gadget platform now supports OAuth, and the interoperability of this standard was demonstrated by new iGoogle gadgets that AOL and MySpace both built to enable users to see their respective AOL or MySpace mailboxes (and other information) while on iGoogle. However, to ensure the user's privacy, this only works after the user has authorized AOL or MySpace to make their data available to the gadget running on iGoogle. We also previously announced that third-party developers can build their own iGoogle gadgets that access the OAuth-enabled APIs for Google applications such as Calendar, Picasa, and Docs. In fact, since both the gadget platform and OAuth technology are open standards, we are working to help other companies who run services similar to iGoogle to enhance them with support for these standards. Once that is in place, these new OAuth-powered gadgets that are available on iGoogle will also work on those other sites, including many of the gadgets that Google offers for its own applications. This provides a platform for some interesting mash-ups. For example, a third-party developer could create a single gadget that uses OAuth to access both Google OAuth-enabled APIs (such as a Gmail user's address book) and MySpace OAuth-enabled APIs (such as a user's friend list) and display a mashup of the combination.

While the combination of OAuth with gadgets is an exciting new use of the technology, most of the use of OAuth is between websites, such as to enable a user of Google Health to allow a clinical trial matching site to access his or her health profile. I previously mentioned that one privacy control provided by OAuth is that it defines a standard way for users to authorize one website to make their data accessible to another website. In addition, OAuth provides a way to do this without the first site needing to reveal the identity of the user -- it simply provides a different opaque security token to each additional website the user wants to share his or her data with. It would allow a mutual fund, for example, to provide an iGoogle gadget to their customers that would run on iGoogle and show the user the value of his or her mutual fund, but without giving Google any unique information about the user, such as a social security number or account number. In the future, maybe we will even see industries like banks use standards such as OAuth to allow their customers to authorize utility companies to perform direct debit from the user's bank account without that person having to actually share his or her bank account number with the utility vendor.

The OAuth community is continuing to enhance this standard and is very interested in having more companies engaged with its development. The OAuth.net website has more details about the current standard, and I maintain a website with advanced information about Google's use of OAuth, including work on integrating OAuth with desktop apps, and integrating with federation standards such as OpenID and SAML. If you're interested in engaging with the OAuth community, please get in touch with us.

Malware? We don’t need no stinking malware!

Panayiotis Mavrommatis — Fri, 24 Oct 2008 21:25:00 +0000

Written by Oliver Fisher

"This site may harm your computer"
You may have seen those words in Google search results — but what do they mean? If you click the search result link you get another warning page instead of the website you were expecting. But if the web page was your grandmother's baking blog, you're still confused. Surely your grandmother hasn't been secretly honing her l33t computer hacking skills at night school. Google must have made a mistake and your grandmother's web page is just fine...

I work with the team that helps put the warning in Google's search results, so let me try to explain. The good news is that your grandmother is still kind and loves turtles. She isn't trying to start a botnet or steal credit card numbers. The bad news is that her website or the server that it runs on probably has a security vulnerability, most likely from some out-of-date software. That vulnerability has been exploited and malicious code has been added to your grandmother's website. It's most likely an invisible script or iframe that pulls content from another website that tries to attack any computer that views the page. If the attack succeeds, then viruses, spyware, key loggers, botnets, and other nasty stuff will get installed.

If you see the warning on a site in Google's search results, it's a good idea to pay attention to it. Google has automatic scanners that are constantly looking for these sorts of web pages. I help build the scanners and continue to be surprised by how accurate they are. There is almost certainly something wrong with the website even if it is run by someone you trust. The automatic scanners make unbiased decisions based on the malicious content of the pages, not the reputation of the webmaster.

Servers are just like your home computer and need constant updating. There are lots of tools that make building a website easy, but each one adds some risk of being exploited. Even if you're diligent and keep all your website components updated, your web host may not be. They control your website's server and may not have installed the most recent OS patches. And it's not just innocent grandmothers that this happens to. There have been warnings on the websites of banks, sports teams, and corporate and government websites.

Uh-oh... I need help!
Now that we understand what the malware label means in search results, what do you do if you're a webmaster and Google's scanners have found malware on your site?

There are some resources to help clean things up. The Google Webmaster Central blog has some tips and a quick security checklist for webmasters. Stopbadware.org has great information, and their forums have a number of helpful and knowledgeable volunteers who may be able to help (sometimes I'm one of them). You can also use the Google SafeBrowsing diagnostics page for your site (http://www.google.com/safebrowsing/diagnostic?site=) to see specific information about what Google's automatic scanners have found. If your site has been flagged, Google's Webmaster Tools lists some of the URLs that were scanned and found to be infected.

Once you've cleaned up your website, use Google's Webmaster Tools to request a malware review. The automatic systems will rescan your website and the warning will be removed if the malware is gone.

Advance warning
I often hear webmasters asking Google for advance warning before a malware label is put on their website. When the label is applied, Google usually emails the website owners and then posts a warning in Google's Webmaster Tools. But no warning is given ahead of time - before the label is applied - so a webmaster can't quickly clean up the site before a warning is applied.

But, look at the situation from the user's point of view. As a user, I'd be pretty annoyed if Google sent me to a site it knew was dangerous. Even a short delay would expose some users to that risk, and it doesn't seem justified. I know it's frustrating for a webmaster to see a malware label on their website. But, ultimately, protecting users against malware makes the internet a safer place and everyone benefits, both webmasters and users.

Google's Webmaster Tools has started a test to provide warnings to webmasters that their server software may be vulnerable. Responding to that warning and updating server software can prevent your website from being compromised with malware. The best way to avoid a malware label is to never have any malware on the site!

Reviews
You can request a review via Google's Webmaster Tools and you can see the status of the review there. If you think the review is taking too long, make sure to check the status. Finding all the malware on a site is difficult and the automated scanners are far more accurate than humans. The scanners may have found something you've missed and the review may have failed. If your site has a malware label, Google's Webmaster Tools will also list some sample URLs that have problems. This is not a full list of all of the problem URLs (because that's often very, very long), but it should get you started.

Finally, don't confuse a malware review with a request for reconsideration. If Google's automated scanners find malware on your website, the site will usually not be removed from search results. There is also a different process that removes spammy websites from Google search results. If that's happened and you disagree with Google, you should submit a reconsideration request. But if your site has a malware label, a reconsideration request won't do any good — for malware you need to file a malware review from the Overview page.

How long will a review take?
Webmasters are eager to have a Google malware label removed from their site and often ask how long a review of the site will take. Both the original scanning and the review process are fully automated. The systems analyze large portions of the internet, which is big place, so the review may not happen immediately. Ideally, the label will be removed within a few hours. At its longest, the process should take a day or so.

New spam and virus trends from Enterprise

Panayiotis Mavrommatis — Tue, 12 Aug 2008 21:01:00 +0000

Written by Amanda Kleha, Google Apps Security & Compliance team

The Google Apps Security & Compliance team, which provides email and web security for more than 40,000 companies, regularly tracks trends in spam, viruses, and other threats. Check out some of our latest findings over on the Enterprise blog. Also, on Friday, August 15, at 10:00 am PT, we'll be hosting a webinar on keeping your business safe from web and email threats -- tune in if you'd like to learn more.

Keyczar: Safe and Simple Cryptography

Panayiotis Mavrommatis — Mon, 11 Aug 2008 18:06:00 +0000

Written by Steve Weis

Cryptography is notoriously hard to get right and if improperly used, can create serious security holes. Common mistakes include using the wrong cipher modes or obsolete algorithms, composing primitives in an unsafe manner, hard-coding keys in source code, or failing to anticipate the need for future key rotation. With these risks in mind, we're pleased to announce the open-source release of Keyczar.

Keyczar is a cryptographic toolkit that supports encryption and authentication for both symmetric and public-key algorithms. It addresses some of the aforementioned issues by choosing safe defaults, tagging outputs with key version information, and providing a simple application programming interface. Keyczar's key versioning system makes it easy to rotate and revoke keys, without worrying about backward compatibility or making any changes to source code.

We look forward to working with the open source community and continuing to make cryptography safer and easier to use. To download Keyczar or for more information, please visit our Google Code project and discussion group.

Are you using the latest web browser?

Panayiotis Mavrommatis — Wed, 16 Jul 2008 20:24:00 +0000

Written by Thomas Duebendorfer

In view of mass defacements of hundreds of thousand of web pages - with the intent to misuse them to launch drive-by download attacks - security researchers from ETH Zurich, Google, and IBM Internet Security Systems were interested in looking at the other side of the attack: the web browser. By analyzing the web browser versions seen in visits to Google websites, they have shown that more than 600 million Internet users don't use the latest version of their browser.

Slow migration to latest browser version
The researchers' paper, entitled "Understanding the Web Browser Threat", shows that as of June 2008, only 59.1% percent of Internet users worldwide use the latest major version of their preferred web browser. Firefox users are the most attentive: 92.2% of them surfed with Firefox 2, the latest major version before the recently released 3.0. Only 52.5% of Microsoft Internet Explorer users have updated to version 7, which is the most secure according to multiple publicly-cited Microsoft experts (among them Sandi Hardmeier). The study revealed that 637 million Internet users worldwide who use web browsers are either not running the latest version of their preferred browser or have not installed the latest patches. These users are vulnerable to exploitation due to their web browser's "built-in" vulnerabilities and the lack of more recent security mechanisms such as improved phishing protection.

Neglected security patches
Over the past 18 months, the study also shows, a maximum of 83.3% of Firefox users were using the latest major version of the web browser and also had all current patches installed (i.e. latest minor version). Only 56.1% and 47.6% of Opera and Internet Explorer users, respectively, were similarly utilizing fully-patched web browsers. Apple users are no better: since the public release of Safari 3, only 65.3% of users operate the latest Safari version.

Maximum measured share of users surfing the web with the most secure versions of Firefox, Safari, Opera and Internet Explorer in June 2008 as seen on Google websites.

Obsolete browser warning
The study's most important finding is that technical measures now in place do not sufficiently guarantee browser security, and that users' security awareness must be further developed. The problem is that most users are unaware that they are not using their browser's latest version. It must be made clear to web browser users that outdated software is associated with significantly higher risk. The researchers therefore suggest that, as a critical component of web software, a visible warning be instituted that warns the user of missing security patches in a way analogous to the 'best before' date in the perishable food industry. Software updates must also be made easier to find. The resulting transparency would go far in contributing to end user awareness of software weaknesses, and allow users to better evaluate risks.

Example "best before" implementation on a Web browser

As a side effect, having users migrate faster to the latest browser version would not only increase security but also make the lives of webmasters easier, as they would need to test and optimize websites for fewer older versions of web browsers.

All Your iFrame Are Point to Us

Panayiotis Mavrommatis — Mon, 11 Feb 2008 21:57:00 +0000

Written by Niels Provos, Anti-Malware Team

It has been over a year and a half since we started to identify web pages that infect vulnerable hosts via drive-by downloads, i.e. web pages that attempt to exploit their visitors by installing and running malware automatically. During that time we have investigated billions of URLs and found more than three million unique URLs on over 180,000 web sites automatically installing malware. During the course of our research, we have investigated not only the prevalence of drive-by downloads but also how users are being exposed to malware and how it is being distributed. Our research paper is currently under peer review, but we are making a technical report [PDF] available now. Although our technical report contains a lot more detail, we present some high-level findings here:

Search Results Containing a URL Labeled as Harmful

The above graph shows the percentage of daily queries that contain at least one search result labeled as harmful. In the past few months, more than 1% of all search results contained at least one result that we believe to point to malicious content and the trend seems to be increasing.

Browsing Habits

Good computer hygiene, such as running automatic updates for the operating system and third-party applications, as well as installing anti-virus products goes a long way in protecting your home computer. However, we have been wondering if users' browsing habits impact the likelihood of encountering malicious web pages. To study this aspect, we took a sample of ~7 million URLs and mapped them to DMOZ categories. Although we found that adult web pages may increase the risk of exploitation, each DMOZ category was affected.

Malicious Content Injection

To understand if malicious content on a web server is due to poor web server security, we analyzed the version numbers reported by web servers on which we found malicious pages. Specifically, we looked at the Apache and the PHP versions exported as part of a server's response. We found that over 38% of both Apache and PHP versions were outdated increasing the risk of remote content injection to these servers.

Our "Ghost In the Browser [PDF]" paper highlighted third-party content as one potential vector of malicious content. Today, a lot of third-party content is due to advertising. To assess the extent to which advertising contributes to drive-by downloads, we analyze the distribution chain of malware, i.e. all the intermediary URLs a browser downloads before reaching a malware payload. We inspected each distribution chain for membership in about 2,000 known advertising networks. If any URL in the distribution chain corresponds to a known advertising network, we count the whole page as being infectious due to Ads. In our analysis, we found that on average 2% of malicious web sites were delivering malware via advertising. The underlying problem is that advertising space is often syndicated to other parties who are not known to the web site owner. Although non-syndicated advertising networks such as Google Adwords are not affected, any advertising networks practicing syndication needs to carefully study this problem. Our technical report [PDF] contains more detail including an analysis based on the popularity of web sites.

Structural Properties of Malware Distribution

Finally, we also investigated the structural properties of malware distribution sites. Some malware distribution sites had as many as 21,000 regular web sites pointing to them. We also found that the majority of malware was hosted on web servers located in China. Interestingly, Chinese malware distribution sites are mostly pointed to by Chinese web servers.

We hope that an analysis such as this will help us to better understand the malware problem in the future and allow us to protect users all over the Internet from malicious web sites as best as we can. One thing is clear - we have a lot of work ahead of us.

Auditing open source software

Panayiotis Mavrommatis — Mon, 08 Oct 2007 23:13:00 +0000

Written by Chris Evans, Security Team

Google encourages its employees to contribute back to the open source community, and there is no exception in Google's Security Team. Let's look at some interesting open source vulnerabilities that were located and fixed by members of Google's Security team. It is interesting to classify and aggregate the code flaws leading to the vulnerabilities, to see if any particular type of flaw is more prevalent.

JDK. In May 2007, I released details on an interesting bug in the ICC profile parser in Sun's JDK. The bug is particularly interesting because it could be exploited by an evil image. Most previous JDK bugs involve a user having to run a whole evil applet. The key parts of code which demonstrate the bug are as follows:
TagOffset = SpGetUInt32 (&Ptr); if (ProfileSize < TagOffset) return SpStatBadProfileDir; ... TagSize = SpGetUInt32 (&Ptr); if (ProfileSize < TagOffset + TagSize) return SpStatBadProfileDir; ... Ptr = (KpInt32_t *) malloc ((unsigned int)numBytes+HEADER);

Both TagSize and TagOffset are untrusted unsigned 32-bit values pulled out of images being parsed. They are added together, causing a classic integer overflow condition and the bypass of the size check. A subsequent additional integer overflow in the allocation of a buffer leads to a heap-based buffer overflow.

gunzip. In September 2006, my colleague Tavis Ormandy reported some interesting vulnerabilities in the gunzip decompressor. They were triggered when an evil compressed archive is decompressed. A lot of programs will automatically pass compressed data through gunzip, making it an interesting attack. The key parts of the code which demonstrate one of the bugs are as follows:
ush count[17], weight[17], start[18], *p; ... for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++;

Here, the stack-based array "count" is indexed by values in the "bitlen" array. These values are under the control of data in the incoming untrusted compressed data, and were not checked for being within the bounds of the "count" array. This led to corruption of data on the stack.

libtiff. In August 2006, Tavis reported a range of security vulnerabilities in the libtiff image parsing library. A lot of image manipulation programs and services will be using libtiff if they handle TIFF format files. So, an evil TIFF file could compromise a lot of desktops or even servers. The key parts of the code which demonstrate one of the bugs are as follows:
if (sp->cinfo.d.image_width != segment_width || sp->cinfo.d.image_height != segment_height) { TIFFWarningExt(tif->tif_clientdata, module, "Improper JPEG strip/tile size, expected %dx%d, got %dx%d",

Here, a TIFF file containing a JPEG image is being processed. In this case, both the TIFF header and the embedded JPEG image contain their own copies of the width and height of the image in pixels. This check above notices when these values differ, issues a warning, and continues. The destination buffer for the pixels is allocated based on the TIFF header values, and it is filled based on the JPEG values. This leads to a buffer overflow if a malicious image file contains a JPEG with larger dimensions than those in the TIFF header. Presumably the intent here was to support broken files where the embedded JPEG had smaller dimensions than those in the TIFF header. However, the consequences of larger dimensions that those in the TIFF header had not been considered.

We can draw some interesting conclusions from these bugs. The specific vulnerabilities are integer overflows, out-of-bounds array accesses and buffer overflows. However, the general theme is using an integer from an untrusted source without adequately sanity checking it. Integer abuse issues are still very common in code, particular code which is decoding untrusted binary data or protocols. We recommend being careful using any such code until it has been vetted for security (by extensive code auditing, fuzz testing, or preferably both). It is also important to watch for security updates for any decoding software you use, and keep patching up to date.

Malware reviews via Webmaster Tools

Panayiotis Mavrommatis — Mon, 13 Aug 2007 19:27:00 +0000

Written by Panayiotis Mavrommatis, Anti-Malware Team

In the past year, the number of sites affected by malware/badware grew from a handful a week to thousands per week. We noted your suggestions to improve communication for webmasters of affected sites -- suggestions mentioned in our earlier blog post "About badware warnings" as well as the stopbadware discussion group. Now, Webmaster Tools provides malware reviews.

If you find that your site is affected by malware, either through malware-labeled search results or in the summary for your site in Webmaster Tools, we've streamlined the process to review your site and return it malware-label-free in our search results:

View a sample of the dangerous URLs on your site in Webmaster Tools.
Make any necessary changes to your site according to StopBadware.org's Security tips.
New: Request a malware review from Google and we'll evaluate your site.
New: Check the status of your review.

If we feel the site is still harmful, we'll provide an updated list of remaining dangerous URLs
If we've determined the site to be clean, you can expect removal of malware messages in the near future (usually within 24 hours).

We encourage all webmasters to become familiar with Stopbadware's malware prevention tips. If you have additional questions, please review our documentation or post to the discussion group. We hope you find this new feature in Webmaster Tools useful in discovering and fixing any malware-related problems, and thanks for your diligence for awareness and prevention of malware.

Automating web application security testing

Panayiotis Mavrommatis — Mon, 16 Jul 2007 18:40:00 +0000

Posted by Srinath Anantharaju, Security Team

Cross-site scripting (aka XSS) is the term used to describe a class of security vulnerabilities in web applications. An attacker can inject malicious scripts to perform unauthorized actions in the context of the victim's web session. Any web application that serves documents that include data from untrusted sources could be vulnerable to XSS if the untrusted data is not appropriately sanitized. A web application that is vulnerable to XSS can be exploited in two major ways:

Stored XSS - Commonly exploited in a web application where one user enters information that's viewed by another user. An attacker can inject malicious scripts that are executed in the context of the victim's session. The exploit is triggered when a victim visits the website at some point in the future, such as through improperly sanitized blog comments and guestbook entries, which facilitates stored XSS.

Reflected XSS - An application that echoes improperly sanitized user input received as query parameters is vulnerable to reflected XSS. With a vulnerable application, an attacker can craft a malicious URL and send it to the victim via email or any other mode of communication. When the victim visits the tampered link, the page is loaded along with the injected script that is executed in the context of the victim's session.

The general principle behind preventing XSS is the proper sanitization (via, for instance, escaping or filtering) of all untrusted data that is output by a web application. If untrusted data is output within an HTML document, the appropriate sanitization depends on the specific context in which the data is inserted into the HTML document. The context could be in the regular HTML body, tag attributes, URL attributes, URL query string attributes, style attributes, inside JavaScript, HTTP response headers, etc.

The following are some (by no means complete) examples of XSS vulnerabilities. Let's assume there is a web application that accepts user input as the 'q' parameter. Untrusted data coming from the attacker is marked in red.

Injection in regular HTML body - angled brackets not filtered or escaped

Your query '' returned xxx results

Injection inside tag attributes - double quote not filtered or escaped

blah">">

Injection inside URL attributes - non-http(s) URL

javascript:evil_script()">...

In JavaScript context - single quote not filtered or escaped

In the cases where XSS arises from meta characters being inserted from untrusted sources into an HTML document, the issue can be avoided either by filtering/disallowing the meta characters, or by escaping them appropriately for the given HTML context. For example, the HTML meta characters <, >, &, " and ' must be replaced with their corresponding HTML entity references <, >, &, " and ' respectively. In a JavaScript-literal context, inserting a backslash in front of \, ', " and converting the carriage returns, line-feeds and tabs into \r, \n and \t respectively should avoid untrusted meta characters being interpreted as code.

How about an automated tool for finding XSS problems in web applications? Our security team has been developing a black box fuzzing tool called Lemon (deriving from the commonly-recognized name for a defective product). Fuzz testing (also referred to as fault-injection testing) is an automated testing approach based on supplying inputs that are designed to trigger and expose flaws in the application. Our vulnerability testing tool enumerates a web application's URLs and corresponding input parameters. It then iteratively supplies fault strings designed to expose XSS and other vulnerabilities to each input, and analyzes the resulting responses for evidence of such vulnerabilities. Although it started out as an experimental tool, it has proved to be quite effective in finding XSS problems. Besides XSS, it finds other security problems such as response splitting attacks, cookie poisoning problems, stacktrace leaks, encoding issues and charset bugs. Since the tool is homegrown it is easy to integrate into our automated test environment and to extend based on specific needs. We are constantly in the process of adding new attack vectors to improve the tool against known security problems.

Update:
I wanted to respond to a few questions that seem to be common among readers. I've listed them below. Thanks for the feedback. Please keep the questions and comments coming.

Q. Does Google plan to market it at some point?
A. Lemon is highly customized for Google apps and we have no plans of releasing it in near future.

Q. Did Google's security team check out any commercially available fuzzers? Is the ability to keep improving the fuzzer the main draw of a homegrown tool?
A. We did evaluate commercially available fuzzers but felt that our specialized needs could be served best by developing our own tools.