Ehrlich was born 14 March 1854 in Strehlen in the Prussian province of Lower Silesia in what is now south-west Poland. He was the second child of Rosa (Weigert) and Ismar Ehrlich, the leader of the local Jewish community. His father was an innkeeper and distiller of liqueurs and the royal lottery collector in Strehlen, a town of some 5,000 inhabitants. His grandfather, Heymann Ehrlich, had been a fairly successful distiller and tavern manager. Ehrlich was the uncle of Fritz Weigert and cousin of Karl Weigert.

After elementary school, Paul attended the time-honored secondary school Maria-Magdalenen-Gymnasium in Breslau, where he met Albert Neisser, who later became a professional colleague. As a schoolboy (inspired by his cousin Karl Weigert who owned one of the first microtomes), he became fascinated by the process of staining microscopic tissue substances. He retained that interest during his subsequent medical studies at the universities of Wroclaw, Strasbourg, Freiburg im Breisgau and Leipzig. After obtaining his doctorate in 1882, he worked at the Charité in Berlin as an assistant medical director under Theodor Frerichs, the founder of experimental clinical medicine, focusing on histology, hematology and color chemistry (dyes).

He married Hedwig Pinkus (1864–1948) in 1883 in the synagogue in Neustadt (now Prudnik, Poland). The couple had two daughters, Stephanie and Marianne. Hedwig was a sister of Max Pinkus, who was an owner of the textile factory in Neustadt (later known as ZPB "Frotex"). He settled in the villa of the Fränkel family on Wiesenerstrasse in Neustadt.

In the early 1870s, Ehrlich's cousin Karl Weigert was the first person to stain bacteria with dyes and to introduce aniline pigments for histological studies and bacterial diagnostics. During his studies in Strassburg under the anatomist Heinrich Wilhelm Waldeyer, Ehrlich continued the research started by his cousin in pigments and staining tissues for microscopic study. He spent his eighth university semester in Freiburg im Breisgau investigating primarily the red dye dahlia (monophenylrosanilin), giving rise to his first publication.

In 1878 he followed his dissertation supervisor Julius Friedrich Cohnheim to Leipzig, and that year obtained a doctorate with a dissertation entitled "Contributions to the Theory and Practice of Histological Staining" (Beiträge zur Theorie und Praxis der histologischen Färbung).

One of the most outstanding results of his dissertation investigations was the discovery of a new cell type. Ehrlich discovered in the protoplasm of supposed plasma cells a granulate which could be made visible with the help of an alkaline dye. He thought this granulate was a sign of good nourishment, and accordingly named these cells mast cells, (from the German word for an animal-fattening feed, Mast). This focus on chemistry was unusual for a medical dissertation. In it, Ehrlich presented the entire spectrum of known staining techniques and the chemistry of the pigments employed. While he was at the Charité, Ehrlich elaborated upon the differentiation of white blood cells according to their different granules. A precondition was a dry specimen technique, which he also developed. A drop of blood placed between two glass slides and heated over a Bunsen burner fixed the blood cells while still allowing them to be stained. Ehrlich used both alkaline and acid dyes, and also created new "neutral" dyes. For the first time this made it possible to differentiate the lymphocytes among the leucocytes (white blood cells). By studying their granulation he could distinguish between nongranular lymphocytes, mono- and poly-nuclear leucocytes, eosinophil granulocytes, and mast cells.

When a student in Breslau, Ehrlich was given an opportunity by the pathologist Julius Friedrich Cohnheim to conduct extensive research and was also introduced to Robert Koch, who was at the time a district physician in Wollstein, Posen Province. In his spare time, Koch had clarified the life cycle of the anthrax pathogen and had contacted Ferdinand Cohn, who was quickly convinced by Koch's work and introduced him to his Breslau colleagues. From 30 April to 2 May 1876, Koch presented his investigations in Breslau, which the student Paul Ehrlich was able to attend.

On 24 March 1882, Ehrlich was present when Robert Koch, working since 1880 at the Imperial Public Health Office (Kaiserliches Gesundheitsamt) in Berlin, presented the lecture in which he reported how he was able to identify the tuberculosis pathogen. Ehrlich later described this lecture as his "greatest experience in science". The day after Koch's lecture, Ehrlich had already made an improvement to Koch's staining method, which Koch unreservedly welcomed. From this date on, the two men were bound in friendship.