The building that now houses the Scharf-Gerstenberg Collection was once used as officer barracks and later as a police station. In 1967 it was the Egyptian Museum, which later moved back to Berlin's Museum Island in 2005. It reopened in the summer of 2008 as the Scharf-Gerstenberg Collection.
The collection began with Otto Gerstenberg who built up one of the most impressive art collections in Germany. His collection included almost all prints made by Albrecht Dürer, Francisco de Goya and Rembrandt. He also collected paintings from Claude Monet, Alfred Sisley, Auguste Renoir and Edgar Degas. Gerstenberg eventually owned the complete set of prints of Henri Toulouse-Lautrec.
Unfortunately his collection was severely damaged during World War II. In 1961, the pieces that remained were handed down to his grandsons Walther and Dieter Scharf, the latter of whom inherited prints by Francisco de Goya and Charles Meryon, as well as a lithograph series by Édouard Manet. His own collection included 300 pieces from over 50 Surrealist and Symbolist artists such as Paul Klee, Pablo Picasso and Alberto Giacometti. Other artists on display include Odilon Redon, Max Klinger, Alfred Kubin, Salvador Dalí, Max Ernst and René Magritte.