Getting to Know a PhD
November 26th, 2014 | Published in Google Student Blog
Google offers a variety of opportunities for PhD students who wish to gain industry experience. Through our Getting to Know a PhD series, we’ll provide a glimpse into some of these opportunities as well as the impactful projects PhD students at Google work on.
Today we’re featuring Neil Houlsby, a former Google European Doctoral Fellowship recipient who interned on the Natural Language Processing research team.
So Neil, tell us about yourself and your PhD topic ...
I took an engineering degree at the University of Cambridge. I stayed on at Cambridge to do a masters in machine learning and cognitive science in the Computational and Biological Learning Lab, supervised by Máté Lengyel, then a PhD in machine learning co-supervised by Zoubin Ghahramani and Máté.
My PhD topic was in statistical machine learning, covering broadly two themes, active learning and matrix modelling. Active learning, or experimental design, involves designing algorithms that automatically choose the best data to collect. This is important when data is scarce or expensive, so minimizing redundancy is essential. One interesting application that I looked at was quantum tomographical experiments. Here, one wishes to characterise a quantum state efficiently; the active learning algorithm adapts the configuration of the experimental apparatus on-the-fly to maximize knowledge about the unknown state. My work on matrices involved designing probabilistic models and scalable algorithms to learn from matrix data, such as online purchasing data, networks, or psychometric questionnaires. One can exploit learned patterns to predict future behaviour, or infer people’s personality traits. I was lucky enough to be involved in a number of other collaborations, and the unifying theme of my PhD was the application Bayesian machine learning and scalable learning algorithms.
Why did you apply for an internship at Google and how supportive was your PhD advisor?
Statistical machine learning is an exciting field because there is much interacting research between theory and applications. In Zoubin’s lab we had a fantastic exposure to the statistical aspects of machine learning. Industrial experience allowed me to work more on large scale applications, but using similar statistical learning techniques that I was working on at Cambridge. My advisor, Zoubin, was extremely encouraging of my internship and other academic visits to gain new experiences in machine learning - provided that I finished my degree on time!
What project was your internship focused on?
I worked on semantic understanding. The goal was to annotate text with its referent entities (anything with a Wikipedia article) e.g. ‘Croft scored a century’ is referring to Croft the cricket player, not the fictional character, and ‘century’ means 100 runs, not a period of time. The algorithm needs to learn how to use context to disambiguate the annotation. Unlike previous approaches, we framed this as an inference problem in a probabilistic model. As well as the modelling aspects, much of the research focussed on how to do learning with a ‘Google scale’ model and perform efficient reasoning over millions of possible entities.
Did you publish at Google during your internship?
Yes, we published the project at the 2014 European Conference on Information Retrieval. This conference is not one that the Cambridge lab usually participated in, so attending and presenting my internship work here was useful to broaden the reach of my research.
How closely connected was the work you did during your internship to your PhD topic?
There was a substantial overlap in the machine learning methods used in my internship work and my PhD (topic modelling, variational Bayes, sampling), but my internship was a stand-alone project that did not overlap directly with my other research. For me, a novel (and fun) part of the internship was working with the Google infrastructure and computing clouds which, naturally, is harder to do outside the company.
What impact has this internship experience had on your PhD?
There were two main impacts. Firstly, I learned from my intern host, Massimiliano Ciaramita, and colleagues at Google a great deal about applied machine learning and more broadly, other topics in computer science. Some techniques I learned were directly applicable during my PhD, others added to my general academic education. Secondly, by broadening my view of machine learning, the internship fuelled my enthusiasm for the field, which motivated me during my PhD and beyond.
Has this internship experience impacted the way you think about your future career?
I always expected that I would pursue a career in computer science and research. I don’t think this has changed. However, the internship revealed the possibility of doing fascinating research in industry. It was only after my internship that I seriously considered a career in industry. Although I sometimes considered very different career paths, from my perspective academic research and research/engineering at Google have many similar challenges and possibilities.
You just recently started your job as a Research Scientist on the Pragmatics team in Zurich - What are you working on now?
I continue working in Natural Language Processing, but I am in a new research team, focussing on pragmatics, discourse and dialogue. Our team consists of a mixture of researchers with backgrounds in linguistics, NLP and machine learning. This is a fun and new research area of for me, I am continuing to use machine learning in much of my work, and am enjoying applying it to this rich and rapidly developing field.
Looking back on your experiences now: Why should a PhD student apply for an internship at Google? Any advice to offer?
When doing research at a particular university you tend to get a single view of your field of interest. It is definitely worth visiting industry and other institutions to broaden your field of view. A Google internship provides a unique research experience: the opportunity to work on some of the hardest problems at the largest possible scales, not to mention the unique environment and culture. Whether you decide to go into industry, or continue in academia, you can learn a great deal during an internship, and have a lot of fun. I would advise applying early in your PhD, as it only gets harder later in your PhD to find the time for an internship. Also, take the opportunity to do something new - apply to work in a different country, or work in a different aspect of your field. A PhD is a unique time when you have the flexibility to explore future possibilities, so take the opportunity while you can.
Today we’re featuring Neil Houlsby, a former Google European Doctoral Fellowship recipient who interned on the Natural Language Processing research team.
I took an engineering degree at the University of Cambridge. I stayed on at Cambridge to do a masters in machine learning and cognitive science in the Computational and Biological Learning Lab, supervised by Máté Lengyel, then a PhD in machine learning co-supervised by Zoubin Ghahramani and Máté.
My PhD topic was in statistical machine learning, covering broadly two themes, active learning and matrix modelling. Active learning, or experimental design, involves designing algorithms that automatically choose the best data to collect. This is important when data is scarce or expensive, so minimizing redundancy is essential. One interesting application that I looked at was quantum tomographical experiments. Here, one wishes to characterise a quantum state efficiently; the active learning algorithm adapts the configuration of the experimental apparatus on-the-fly to maximize knowledge about the unknown state. My work on matrices involved designing probabilistic models and scalable algorithms to learn from matrix data, such as online purchasing data, networks, or psychometric questionnaires. One can exploit learned patterns to predict future behaviour, or infer people’s personality traits. I was lucky enough to be involved in a number of other collaborations, and the unifying theme of my PhD was the application Bayesian machine learning and scalable learning algorithms.
Why did you apply for an internship at Google and how supportive was your PhD advisor?
Statistical machine learning is an exciting field because there is much interacting research between theory and applications. In Zoubin’s lab we had a fantastic exposure to the statistical aspects of machine learning. Industrial experience allowed me to work more on large scale applications, but using similar statistical learning techniques that I was working on at Cambridge. My advisor, Zoubin, was extremely encouraging of my internship and other academic visits to gain new experiences in machine learning - provided that I finished my degree on time!
What project was your internship focused on?
I worked on semantic understanding. The goal was to annotate text with its referent entities (anything with a Wikipedia article) e.g. ‘Croft scored a century’ is referring to Croft the cricket player, not the fictional character, and ‘century’ means 100 runs, not a period of time. The algorithm needs to learn how to use context to disambiguate the annotation. Unlike previous approaches, we framed this as an inference problem in a probabilistic model. As well as the modelling aspects, much of the research focussed on how to do learning with a ‘Google scale’ model and perform efficient reasoning over millions of possible entities.
Did you publish at Google during your internship?
Yes, we published the project at the 2014 European Conference on Information Retrieval. This conference is not one that the Cambridge lab usually participated in, so attending and presenting my internship work here was useful to broaden the reach of my research.
How closely connected was the work you did during your internship to your PhD topic?
There was a substantial overlap in the machine learning methods used in my internship work and my PhD (topic modelling, variational Bayes, sampling), but my internship was a stand-alone project that did not overlap directly with my other research. For me, a novel (and fun) part of the internship was working with the Google infrastructure and computing clouds which, naturally, is harder to do outside the company.
What impact has this internship experience had on your PhD?
There were two main impacts. Firstly, I learned from my intern host, Massimiliano Ciaramita, and colleagues at Google a great deal about applied machine learning and more broadly, other topics in computer science. Some techniques I learned were directly applicable during my PhD, others added to my general academic education. Secondly, by broadening my view of machine learning, the internship fuelled my enthusiasm for the field, which motivated me during my PhD and beyond.
Has this internship experience impacted the way you think about your future career?
I always expected that I would pursue a career in computer science and research. I don’t think this has changed. However, the internship revealed the possibility of doing fascinating research in industry. It was only after my internship that I seriously considered a career in industry. Although I sometimes considered very different career paths, from my perspective academic research and research/engineering at Google have many similar challenges and possibilities.
You just recently started your job as a Research Scientist on the Pragmatics team in Zurich - What are you working on now?
I continue working in Natural Language Processing, but I am in a new research team, focussing on pragmatics, discourse and dialogue. Our team consists of a mixture of researchers with backgrounds in linguistics, NLP and machine learning. This is a fun and new research area of for me, I am continuing to use machine learning in much of my work, and am enjoying applying it to this rich and rapidly developing field.
Looking back on your experiences now: Why should a PhD student apply for an internship at Google? Any advice to offer?
When doing research at a particular university you tend to get a single view of your field of interest. It is definitely worth visiting industry and other institutions to broaden your field of view. A Google internship provides a unique research experience: the opportunity to work on some of the hardest problems at the largest possible scales, not to mention the unique environment and culture. Whether you decide to go into industry, or continue in academia, you can learn a great deal during an internship, and have a lot of fun. I would advise applying early in your PhD, as it only gets harder later in your PhD to find the time for an internship. Also, take the opportunity to do something new - apply to work in a different country, or work in a different aspect of your field. A PhD is a unique time when you have the flexibility to explore future possibilities, so take the opportunity while you can.