In early times, the Chinese name for Stonecutters Island (昂船洲) was written as ‘仰船洲’. In the 1860s, the government decided to build a prison on the island, to relieve the over-crowding of Victoria Prison in Central. However, in 1874, the prison was destroyed by a typhoon, leaving Stonecutters Island as a deserted island.

In 1883, Stonecutters Island became a military base, mainly used as an arsenal; a barracks and a sanatorium were also built. In 1889, the Hong Kong Government legislated the whole island as a restricted area, and access by civilians and vessels was prohibited. During the Second World War, a series of military facilities were built on the island to resist the Japanese army. Before the Japanese occupation of Hong Kong, troops were stationed on Stonecutters Island. They were repeatedly attacked by the Japanese army, and eventually had to retreat from the island. The place was used as a Japanese military base during the Japanese occupation of Hong Kong. After the war, Stonecutters Island continued to be reserved for military use. Since 1997, it has been a base for the People's Liberation Army Hong Kong Garrison. Since the 1990s, due to extensive reclamation, Stonecutters Island became an integral part of the Kowloon Peninsula.