The land on which Jerusalem sits, as well as the city itself, has historically had great significance for the Jews because of the important events that took place there. The patriarch of the Jews, Abraham, was asked by God to sacrifice his only son on Mount Moriah, which is in present-day Jerusalem.
Later, during Moses' time, God helped Moses to remove the children of Israel (the Jews) from bondage in Egypt, and God promised the Jews a promised land, flowing with milk and honey. Those who left Egypt had some crises of faith that resulted in a long journey through the wilderness, but eventually they were commanded to go into the land that would become Israel (and Jerusalem would become capital). Some time after the children of Israel took over the land, God gave them King David, who loved God and wanted to build God a temple in Jerusalem in which people could worship Him. However, it would be David's son, Solomon, who would build the temple. God came to dwell in the part of the temple called the "Holy of Holies." As with the time they were in Egypt, the children of Israel had more crises in faith, often abandoning God for the local deities and not following the commandments God had given them. As a result, God sent them into captivity in Babylon for 70 years. Eventually they came back to Jerusalem and rebuilt the temple, which had been destroyed. The rebuilding was not nearly as nice as the original temple, but it stood until about 70 years after the birth of Jesus Christ, when the Romans destroyed it. The Old Testament of the Bible describes all these and other events, which demonstrate why Jerusalem is so important to the Jews.