This book is organized in two parts. Part I, The Fundamentals of Machine Learning, covers the following topics:
What is Machine Learning? What problems does it try to solve? What are the main categories and fundamental concepts of Machine Learning systems?
The main steps in a typical Machine Learning project.
Learning by fitting a model to data.
Optimizing a cost function.
Handling, cleaning, and preparing data.
Selecting and engineering features.
Selecting a model and tuning hyperparameters using cross-validation.
The main challenges of Machine Learning, in particular underfitting and overfitting (the bias/variance tradeoff).
Reducing the dimensionality of the training data to fight the curse of dimensionality.
The most common learning algorithms: Linear and Polynomial Regression, Logistic Regression, kNearest Neighbors, Support Vector Machines, Decision Trees, RandomForests, and Ensemble methods.
Part II, Neural Networks and Deep Learning, covers the following topics:
What are neural nets? What are they good for?
Building and training neural nets using TensorFlow.
The most important neural net architectures: feedforward neural nets, convolutional nets, recurrent nets, long short-termmemory (LSTM) nets, and autoencoders.
Techniques for training deep neural nets.
Scaling neural networks for huge datasets.
Reinforcement learning.
The first part is based mostly on Scikit-Learn while the second part uses TensorFlow.
Don’t jump into deep waters too hastily: while Deep Learning is no doubt one of the most exciting areas in Machine Learning, you should master the fundamentals first Moreover, most problems can be solved quite well using simpler techniques such as Random Forests and Ensemble methods (discussed in Part I). Deep Learning is best suited for complex problems such as image recognition, speech recognition, or natural language processing, provided you have enough data, computing power, and patience.
Using Code Examples
Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/ageron/handson-ml.
I amincredibly grateful to all the amazing people who took time out of their busy lives to review my book in so much detail. Thanks to Pete Warden for answering all my TensorFlow questions, reviewing Part II, providing many interesting insights, and of course for being part of the core TensorFlow team. You should definitely check out his blog! Many thanks to Lukas Biewald for his very thorough review of Part II: he left no stone unturned, tested all the code (and caught a few errors), made many great suggestions, and his enthusiasmwas contagious. You should check out his blog and his cool robots! Thanks to Justin Francis, who also reviewed Part II very thoroughly, catching errors and providing great insights, in particular in Chapter 16. Check out his posts on TensorFlow!
And of course, a gigantic “thank you” to my dear brother Sylvain, who reviewed every single chapter, tested every line of code, provided feedback on virtually every section, and encouraged me fromthe first line to the last. Love you, bro!
Many thanks as well to O’Reilly’s fantastic staff, in particular Nicole Tache, who gave me insightful feedback, always cheerful, encouraging, and helpful. Thanks as well to Marie Beaugureau, Ben Lorica, Mike Loukides, and Laurel Ruma for believing in this project and helping me define its scope. Thanks to Matt Hacker and all of the Atlas teamfor answering all my technical questions regarding formatting, asciidoc, and LaTeX, and thanks to Rachel Monaghan, Nick Adams, and all of the production teamfor their final review and their hundreds of corrections.
Machine Learning is the science (and art) of programming computers so they can learn from data. Here is a slightly more general definition:
[Machine Learning is the] field of study that gives computers the ability to learn without being explicitly programmed.
Arthur Samuel, 1959
And a more engineering-oriented one:
Acomputer programis said to learn fromexperience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. TomMitchell, 1997
For example, your spamfilter is a Machine Learning programthat can learn to flag spamgiven examples of spamemails (e.g., flagged by users) and examples of regular (nonspam, also called “ham”) emails. The examples that the systemuses to learn are called the training set. Each training example is called a training instance (or sample). In this case, the task T is to flag spamfor new emails, the experience E is the training data, and the performance measure P needs to be defined; for example, you can use the ratio of correctly classified emails. This particular performance measure is called accuracy and it is often used in classification tasks.
If you just download a copy of Wikipedia, your computer has a lot more data, but it is not suddenly better at any task. Thus, it is not Machine Learning.
1-3 Automatically adapting to change
Another area where Machine Learning shines is for problems that either are too complex for traditional approaches or have no known algorithm. For example, consider speech recognition: say you want to start simple and write a programcapable of distinguishing the words “one” and “two.” You might notice that the word “two” starts with a high-pitch sound (“T”), so you could hardcode an algorithmthat measures high-pitch sound intensity and use that to distinguish ones and twos. Obviously this technique will not scale to thousands of words spoken by millions of very different people in noisy environments and in dozens of languages. The best solution (at least today) is to write an algorithmthat learns by itself, given many example recordings for each word.
Finally, Machine Learning can help humans learn (Figure 1-4): MLalgorithms can be inspected to see what they have learned (although for some algorithms this can be tricky). For instance, once the spam filter has been trained on enough spam, it can easily be inspected to reveal the list of words and combinations of words that it believes are the best predictors of spam. Sometimes this will reveal unsuspected correlations or new trends, and thereby lead to a better understanding of the problem.
Applying MLtechniques to dig into large amounts of data can help discover patterns that were not immediately apparent. This is called data mining.
Figure
1-4. Machine Learning can help humans learn
To summarize, Machine Learning is great for:
Problems for which existing solutions require a lot of hand-tuning or long lists of rules: one Machine Learning algorithmcan often simplify code and performbetter.
Complex problems for which there is no good solution at all using a traditional approach: the best Machine Learning techniques can find a solution.
Fluctuating environments: a Machine Learning systemcan adapt to new data.
Getting insights about complex problems and large amounts of data.
Figure
Types of Machine Learning Systems
There are so many different types of Machine Learning systems that it is useful to classify themin broad categories based on:
Whether or not they are trained with human supervision (supervised, unsupervised, semisupervised, and Reinforcement Learning)
Whether or not they can learn incrementally on the fly (online versus batch learning)
Whether they work by simply comparing new data points to known data points, or instead detect patterns in the training data and build a predictive model, much like scientists do (instance-based versus model-based learning)
These criteria are not exclusive; you can combine themin any way you like. For example, a state-of-theart spamfilter may learn on the fly using a deep neural network model trained using examples of spamand ham; this makes it an online, model-based, supervised learning system.
Let’s look at each of these criteria a bit more closely.
Discovering Diverse Content Through Random Scribd Documents
CHAPTER XI.
GENERAL TOM THUMB IN ENGLAND.
ARRIVAL IN LONDON THE GENERAL’S DEBUT IN THE PRINCESS’S THEATRE ENORMOUS SUCCESS MY MANSION AT THE WEST END DAILY LEVEES FOR THE NOBILITY AND GENTRY HON EDWARD EVERETT HIS INTEREST IN THE GENERAL VISIT TO THE BARONESS ROTHSCHILD OPENING IN EGYPTIAN HALL, PICCADILLY MR CHARLES MURRAY, MASTER OF THE QUEEN’S HOUSEHOLD AT BUCKINGHAM PALACE BY COMMAND OF HER MAJESTY A ROYAL RECEPTION THE FAVORABLE IMPRESSION MADE BY THE GENERAL AMUSING INCIDENTS OF THE VISIT BACKING OUT FIGHT WITH A POODLE COURT JOURNAL NOTICE SECOND VISIT TO THE QUEEN THE PRINCE OF WALES AND PRINCESS ROYAL THE QUEEN OF THE BELGIANS THIRD VISIT TO BUCKINGHAM PALACE KING LEOPOLD, OF BELGIUM ASSURED SUCCESS THE BRITISH PUBLIC EXCITED EGYPTIAN HALL CROWDED QUEEN DOWAGER ADELAIDE THE GENERAL’S WATCH NAPOLEON AND THE DUKE OF WELLINGTON DISTINGUISHED FRIENDS.
IMMEDIATELY after our arrival in London, the General came out at the Princess’s Theatre, and made so decided a “hit” that it was difficult to decide who was best pleased, the spectators, the manager, or myself. The spectators were delighted because they could not well help it; the manager was satisfied because he had coined money by the engagement; and I was greatly pleased because I now had a visible guaranty of success in London. I was offered far higher terms for a re-engagement, but my purpose had been already answered; the news was spread everywhere that General Tom Thumb, an unparalleled curiosity, was in the city; and it only remained for me to bring him before the public, on my own account and in my own time and way.
I took a furnished mansion in Grafton Street, Bond Street, West End, in the very centre of the most fashionable locality. The house had previously been occupied for several years by Lord Talbot, and Lord Brougham and half a dozen families of the aristocracy and many of the gentry were my
this remarkable specimen of humanity so much smaller than they had evidently expected to find him.
The General advanced with a firm step, and as he came within hailing distance made a very graceful bow, and exclaimed, “Good evening, Ladies and Gentlemen!”
A burst of laughter followed this salutation. The Queen then took him by the hand, led him about the gallery, and asked him many questions, the answers to which kept the party in an uninterrupted strain of merriment. The General familiarly informed the Queen that her picture gallery was “first-rate,” and told her he should like to see the Prince of Wales. The Queen replied that the Prince had retired to rest, but that he should see him on some future occasion. The General then gave his songs, dances, and imitations, and after a conversation with Prince Albert and all present, which continued for more than an hour, we were permitted to depart.
Before describing the process and incidents of “backing out,” I must acknowledge how sadly I broke through the counsel of the Lord in Waiting. While Prince Albert and others were engaged with the General, the Queen was gathering information from me in regard to his history, etc. Two or three questions were put and answered through the process indicated in my drill. It was a round-about way of doing business not at all to my liking, and I suppose the Lord in Waiting was seriously shocked, if not outraged, when I entered directly into conversation with Her Majesty. She, however, seemed not disposed to check my boldness, for she immediately spoke directly to me in obtaining the information which she sought. I felt entirely at ease in her presence, and could not avoid contrasting her sensible and amiable manners with the stiffness and formality of upstart gentility at home or abroad.
The Queen was modestly attired in plain black, and wore no ornaments. Indeed, surrounded as she was by ladies arrayed in the highest style of magnificence, their dresses sparkling with diamonds, she was the last person whom a stranger would have pointed out in that circle as the Queen of England.
The Lord in Waiting was perhaps mollified toward me when he saw me following his illustrious example in retiring from the royal presence. He was accustomed to the process, and therefore was able to keep somewhat ahead (or rather aback) of me, but even I stepped rather fast for the other
member of the retiring party. We had a considerable distance to travel in that long gallery before reaching the door, and whenever the General found he was losing ground, he turned around and ran a few steps, then resumed the position of “backing out,” then turned around and ran, and so continued to alternate his methods of getting to the door, until the gallery fairly rang with the merriment of the royal spectators. It was really one of the richest scenes I ever saw; running, under the circumstances, was an offence sufficiently heinous to excite the indignation of the Queen’s favorite poodle-dog, and he vented his displeasure by barking so sharply as to startle the General from his propriety. He, however, recovered immediately, and with his little cane commenced an attack on the poodle, and a funny fight ensued, which renewed and increased the merriment of the royal party.
This was near the door of exit. We had scarcely passed into the anteroom, when one of the Queen’s attendants came to us with the expressed hope of Her Majesty that the General had sustained no damage—to which the Lord in Waiting playfully added, that in case of injury to so renowned a personage, he should fear a declaration of war by the United States!
The courtesies of the Palace were not yet exhausted, for we were escorted to an apartment in which refreshments had been provided for us. We did ample justice to the viands, though my mind was rather looking into the future than enjoying the present. I was anxious that the “Court Journal” of the ensuing day should contain more than a mere line in relation to the General’s interview with the Queen, and, on inquiry, I learned that the gentleman who had charge of that feature in the daily papers was then in the Palace. He was sent for by my solicitation, and promptly acceded to my request for such a notice as would attract attention. He even generously desired me to give him an outline of what I sought, and I was pleased to see afterwards, that he had inserted my notice verbatim.
This notice of my visit to the Queen wonderfully increased the attraction of my exhibition and compelled me to obtain a more commodious hall for my exhibition. I accordingly removed to the larger room in the same building, for some time previously occupied by our countryman, Mr. Catlin, for his great Gallery of Portraits of American Indians and Indian Curiosities, all of which remained as an adornment.
On our second visit to the Queen, we were received in what is called the “Yellow Drawing-Room,” a magnificent apartment, surpassing in splendor
vest with fancy-colored embroidery, white silk stockings and pumps, wig, bag-wig, cocked hat, and a dress sword.
“Why, General,” said the Queen Dowager, “I think you look very smart to-day.”
“I guess I do,” said the General complacently.
A large party of the nobility were present. The old Duke of Cambridge offered the little General a pinch of snuff, which, he declined. The General sang his songs, performed his dances, and cracked his jokes, to the great amusement and delight of the distinguished circle of visitors.
“Dear little General,” said the kind-hearted Queen, taking him upon her lap, “I see you have got no watch. Will you permit me to present you with a watch and chain?”
“I would like them very much,” replied the General, his eyes glistening with joy as he spoke.
“I will have them made expressly for you,” responded the Queen Dowager; and at the same moment she called a friend and desired him to see that the proper order was executed. A few weeks thereafter we were called again to Marlborough House. A number of the children of the nobility were present, as well as some of their parents. After passing a few compliments with the General, Queen Adelaide presented him with a beautiful little gold watch, placing the chain around his neck with her own hands. The little fellow was delighted, and scarcely knew how sufficiently to express his thanks. The good Queen gave him some excellent advice in regard to his morals, which he strictly promised to obey.
After giving his performances, we withdrew from the royal presence, and the elegant little watch presented by the hands of Her Majesty the Queen Dowager was not only duly heralded, but was also placed upon a pedestal in the hall of exhibition, together with the presents from Queen Victoria, and covered with a glass vase. These presents, to which were soon added an elegant gold snuff-box mounted with turquoise, presented by his Grace the Duke of Devonshire, and many other costly gifts of the nobility and gentry, added greatly to the attractions of the exhibition. The Duke of Wellington called frequently to see the little General at his public levees. The first time he called, the General was personating Napoleon Bonaparte, marching up and down the platform, and apparently taking snuff in deep meditation. He was dressed in the well-known uniform of the Emperor. I
introduced him to the “Iron Duke,” who inquired the subject of his meditations. “I was thinking of the loss of the battle of Waterloo,” was the little General’s immediate reply. This display of wit was chronicled throughout the country, and was of itself worth thousands of pounds to the exhibition.
While we were in London the Emperor Nicholas, of Russia, visited Queen Victoria, and I saw him on several public occasions. I was present at the grand review of troops in Windsor Park in honor of and before the Emperor of Russia and the King of Saxony.
General Tom Thumb had visited the King of Saxony and also Ibrahim Pacha who was then in London. At the different parties we attended, we met, in the course of the season, nearly all of the nobility. I do not believe that a single nobleman in England failed to see General Tom Thumb at his own house, at the house of a friend, or at the public levees at Egyptian Hall. The General was a decided pet with some of the first personages in the land, among whom may be mentioned Sir Robert and Lady Peel, the Duke and Duchess of Buckingham, Duke of Bedford, Duke of Devonshire, Count d’Orsay, Lady Blessington, Daniel O’Connell, Lord Adolphus Fitzclarence, Lord Chesterfield, Mr. and Mrs. Joshua Bates, of the firm of Baring Brothers &
