Heilongjiang borders Jilin in the south and Inner Mongolia to the west; it also borders Russia to the north.

The Amur River marks the border between the People's Republic of China and Russia to the north. Heilongjiang contains China's northernmost point (in Mohe County along the Amur) and easternmost point (at the junction of the Amur and Ussuri Rivers).

In ancient times Heilongjiang was far from any literate civilization, and information was sparse. Chinese and other sources state that Heilongjiang was inhabited by people such as the Xianbei, the Malgal, and the Khitan. The eastern portion of Heilongjiang was ruled by the kingdom of Balhae between the 7th century and 10th century. The Jurchen Jin Dynasty (1115-1234) that subsequently ruled much of north China arose within the borders of modern Heilongjiang.

Under the Manchu Qing Dynasty, the western part of Heilongjiang was under the supervision of the General of Heilongjiang, whose power extended, according to the Treaty of Nerchinsk, as far north as the Stanovoy Mountains; eastern Heilongjiang was under the supervision of the General of Jilin, whose power reached the Sea of Japan. These areas deep in Manchuria were closed off to Han Chinese migration.

However, in 1858 and 1860 the Qing government gave up all land beyond the Amur and Ussuri Rivers to Russia, cutting China off from the Sea of Japan and giving Heilongjiang its present northern borders. At the same time, Manchuria was opened to Han Chinese migration by the Qing government. By the early twentieth century, the Han Chinese had become the dominant ethnic group in the region. In 1932, present-day Heilongjiang became part of the Japanese puppet state of Manchukuo.

After the Japanese defeat in 1945, Soviet forces entered Manchuria and gave the Chinese communists control over most of the area. Heilongjiang became the first province to be completely controlled by the Chinese communists and Harbin the first major city to be controlled by them. From Manchuria, the communists were able to conduct the initial phases of the Chinese Civil War.

At the beginning of communist rule, Heilongjiang province included only the western portion of the present-day province, and had its capital at Qiqihar. The remaining area was the province of Songjiang; its capital was Harbin. In 1954, these two provinces were merged into present-day Heilongjiang. During the Cultural Revolution Heilongjiang was also expanded to include Hulunbuir League and some other areas previously in Inner Mongolia; this has since mostly been reversed.