It is not a coincidence that the value of Bitcoin increases as mining difficulty rises. It is also not a coincidence that the mining difficulty increases as the value of Bitcoin rises. There is a very tight linkage between the two. To understand why, we must first understand how mining affects the difficulty.
Commercial mining closely resembles regular manufacturing businesses. There are capital costs including mining hardware---ASICs (specialized mining hardware), GPUs (graphics cards), CPUs (regular computers)---and infastructure such as ventilation. Then there is the variable cost of electricity. The goal is to maximize profit: getting the most out of the hardware before it becomes unprofitable and then selling. Most miners sell their newly mined bitcoin immediately to recoup costs, creating market sell pressure and driving down the price. Holders usually do better in the long run by benefitting from the appreciation of bitcoins.
Difficulty increases as more miners deploy more mining hardware. The bitcoin network automatically adjusts the difficulty every 2016 blocks, roughly every two weeks, so that each mined block takes about 10 minutes to find. The current seignorage reward for mining a block is 25 bitcoins. Every machine, even a weak laptop, has a shot at finding the block on a pro rata basis of its hashing power. By joining a mining pool, miners can reduce the variance of the reward similar to office workers buying a batch of lottery tickets to increase the chance of winning, while accepting a smaller reward. In a pool, if one miner finds a block, the reward is distrubed among pool members.
How and why is the bitcoin price correlated with the difficulty?
Every miner looks at the cost of equipment and electricity in order to make two educated guesses. First, "How fast will the difficulty rise?" And second, "How fast will the value of Bitcoin rise?" These two numbers, impossible to know in advance, are critically important. Looking at historical rates and projecting forward provides a method for estimating these numbers.
Efficient market theory says that it will eventually be the same cost to purchase a bitcoin as it will be to mine a bitcoin. Why is this? Because if it’s cheaper to mine a bitcoin than to buy a bitcoin, miners will buy more hardware, burn more electricity, mine, and sell bitcoin into the market pushing the market rate down until mining is no longer profitable. If it is cheaper to buy a bitcoin, then the money that might possibly go into mining will instead purchase bitcoin pushing the market price up.
Why mine at all, if efficient market theory gaurantees a profitless undertaking? Because efficient market theory breaks down at the compressed timescales involved in Bitcoin. Rational market forces have not gone away, but they simply cannot adapt as quickly as the bitcoin ecosystem changes. If you are prepared to mine when the price rockets on speculation, and before the difficulty adjusts to compensate, there are significant profits to be made.
We have all heard stories from the early bitcoin days, literally only a few years ago, when mining bitcoin with a laptop yielded a single coinbase of fifty coins. We have also heard stories of those same people turning off their computers because mining just was not worth it. I want to scream knowing that those same bitcoins are now worth a combined total of tens of thousands of dollars. But those folks were operating logically. The cost of electricity exceeded the value of the bitcoins at the time. It was better to buy bitcoin on the open market with the same money that would have been spent on electricity, although few bought.
So what changed from those easy, breezy, laptop mining days? The difficulty has changed. By design, half of all the bitcoins that will ever be mined were mined in the first four years. Does this mean they were easier to find in the beginning? Yes, but not because they were just laying scattered around somewhere. The careful and considered design of the software by its creator, Satoshi Nakamoto, made them easier to mine at the beginning and harder as more miners join the party. These details are controlled not by elite bureaucrats, but by the bitcoin software, and while software can easily be changed, it is necessary that all miners use software that follows the same rules. Since miners are invested in the current ruleset, it will be nearly impossible to change the protocol by getting most of the miners to switch software.
Now fewer bitcoins are left to mine, and the reduced number of bitcoins are distributed proportionately among the miners based on the resources they marshal for mining. Add more miners and each miner gets fewer bitcoins until some miners drop out because their mining equipment is not as efficient, or their electricity costs are too high. Efficiency is the reason that mining with a laptop is no longer recommended.
Mining is a great way to equitably distribute bitcoin, while performing two other interesting things at the same time. First, mining takes value away from bitcoin by increasing the supply. Second, mining increases the difficulty, making each new bitcoin more expensive to obtain. These two competing forces are in tension.
In summary, value and difficulty are tightly correlated because when the difficulty rises to the point that mining is unprofitable, it makes more sense to purchase bitcoin, which adds buying pressure to the market, and vice versa. The value of a bitcoin is dependent on a high difficulty, and the mining difficulty is dependent on a high price. With a few exceptions, like the recent exponential efficiency gains from improved hardware, the bitcoin value and mining difficulty will both go up, or both go down, but they will not separate, at least for very long.