Societies become breakthrough powerful under fairly specific historical conditions.
Competition in a concentrated area.
This covers most breakthrough shifts. Let’s give some examples.
Europe
A large number of kingdoms and republics, in constant competition. If you didn’t advance militarily, culturally (administration and culture matter) and technologically, you were in trouble. As administration improved and military technology changed to favor “despotic kings” like Louis XIV and Henry the VIIth (a very underrated King), decentralized and smaller power, internal (nobles) and external were brought under control. Constant warfare and other forms of competition lead to rapid advancement.
Fail, and you could fall. If the English hadn’t defeated the Spanish armada, well, that would have been the end of an independent England. Many other principalities did fall.
In addition, there was external pressure, from the more advanced, at least initially, Ottomans, whom the Europeans were terrified of. The Ottoman threat was real, and a few key battles and wars could have swung the other way, and Eastern Europe fallen under Ottoman control.
With no central control of the entirety of Europe, people could move easily, and find a place where whatever new thing they wanted to try was allowed.
The end result was a huge increase in technology, administrative control allowing more and more resources to be brought under central control, and swift advancement in the military. Even before the industrial revolution Europeans wound up conquering a vast chunk of the world, one they had industrialized, the world was at their feet, and they wound up in control of about three-quarters of it, with the rest terrified and compliant. (This is the case with China: never actually conquered, but under the thumb, though they did fight as best they could, they were defeated.)
Ancient Greece, then Rome
Greek city states were in ferocious competition with each other. Militarily, culturally and even technologically. The Greeks were far more advanced than the Romans. If you lost, terrible things could happen, like the destruction of your entire city and the enslavement of every survivor.
The Greeks were also under threat by a great neighbouring power: Persia, and the wars against Persia, were, again, close run. They could have gone the other way. By the time of of the Ten Thousand, when Greek mercenaries who had fought for the losing side in a Persian civil were were able to march across much of a hostile Persian empire, crushing all in their way, it was clear to the Greeks that Persia was ripe—their armies were vastly larger, but the Greek way of war was vastly superior.
Greece itself was conquered by Macedonia, which was essentially Greek, but still somewhat Barbarian, then Macedonia, under Alexander, conquered Persia and Egypt. The Persians, even if their leadership hadn’t been cowardly, never stood a chance. Then the Greeks ruled the Eastern Med and the Near East until the Romans. After Alexander, however, they didn’t expand much. The successor states were not dynamic.
Rome was also in savage competition. Against the Greeks, the other Italian tribes, the Celtic tribes and Carthage. They were almost always at war, and they learned well. Eventually they were able to conquer Greece, Span, most of what is now France, and Egypt, though they never had much luck against most of what had been the Persian Empire. Once the Republic fell, Rome didn’t spread much. The occasional Emperor would conquer some land, but they could rarely hold it. The dynamism of the Republic, and the pressure required for advancement no longer existed. Indeed, as time went by the Romans lost a fair bit of their technology, as well. The big single Empire was not dynamic.
The Mongols
Before Temujin the Mongols and the other steppe nomads near China were in constant competition against each other, and were also constantly subject to manipulation and war from China, which sought to keep them down, fearing (quite rightly) that they would invade. Most of Temujin’s life was spent conquering and unifying the steppe nomads, then changing their culture to be more disciplined and usefully warlike. The steppe nomads had always been fearsome, but Temujin changed how they fought: ordering them in groups of 10, 100, 1,000 and 10,000 and enforcing amazing levels of discipline. The Mongols were not just disciplined in battle, strategically they moved faster than any other military of the time and were able to pull off amazing coordination. Columns of troops would meet hundreds of miles away, on the exact day planned.
Though horse archers are always dangerous, it was Genghis Khan’s unification PLUS his changes to society and military organization which turned them into a terror so great that they were not defeated for over a hundred years. In their time, they were just as dominant as the Europeans in the late 19th century.
There are other examples: the Zulus, who had the bad luck to run into the British at the height of their power are one. Warring States China is another. Ancient India around the time of the Buddha is a third. The Sengoku period of Japan is a fourth, and once Tokugawa took power, Japan, in many ways stagnated. (During the Sengoku period, the Japanese had more firearms than anywhere else.)
Concluding
Large empires are stagnant. There may be some advancement, though often there is none or even retrogression, but they don’t make breakthroughs into revolutionary power. At best they inherit it.
Small groups in competition have the chance, though not the certainty, of fast progress, stuck as they are in a cauldron. It doesn’t always happen: the tribes of New Guinea were caught in zero, indeed, often negative sum competition and remained backwards.
But the general rule of breakout power is small states in serious competition, usually with an outside threat.
