(Bloomberg) — The nodding donkey was invented nearly a century ago, and it’s still hard at work in the oil patch, virtually unchanged, pumping oil out of the ground. There’s been a recent innovation, though: Algorithms adjust the extraction flow based on computer monitoring hundreds of feet below.

Finally. “Onshore North America used to be a market where state-of-the-art technology went to be humiliated,” said Tom Curran, an energy analyst at FBR Capital Markets & Co. “You’ve had a clear shift occur where onshore North America for the first time in recent history has become a technology play.”

The worst crude-market crash in a generation propelled energy companies into the digital world. They had already pretty much tested their physical limits with brute strength, ramping up injections of sand to tease more oil out of subterranean pockets and drilling wells longer and longer. Now they’re using DNA sequencing to track crude molecules and mapping buried streams with imaging software. Robots are fitting pipes together. Roughnecks consult mobile apps for drilling-direction advice. Oilfield services providers find themselves in a new arms race, led by giant Schlumberger Ltd., which recently opened an office on Sand Hill Road in the heart of Silicon Valley.

Real Revolution

It’s a real revolution, because many if not most explorers and drillers were stubbornly slow to look to software for help. They’ve historically been a conservative bunch. And during the good times of $100-a-barrel oil, profits were flowing so fast there wasn’t any incentive to experiment. The price plummeting to $50 since 2014 opened a lot of minds, starting a seismic shift to improve efficiency with data crunching and predictive analytics to get more out of rocks at less cost.

“The promise of oil and gas in the future is to actually start using the data that we have, and then acquire more of it and use more of it,” said Ashok Belani, executive vice president of technology at Schlumberger. “Data in the oil and gas industry has never been organized in the way that a company like Google or Amazon would do it.” And for all the recent progress, “we are in inning one of nine.”

There’s room for improvement in the business of exploration and extraction, particularly in the shale fields that are the most abundant sources left onshore. Recovery rates run about 8%. Getting to the targets requires fracturing oil-soaked rock with injections of water, sand and chemicals. That works about half the time.

Microbes in Rocks

So several companies, including Norway’s Statoil ASA and Houston-based Anadarko Petroleum Corp., have hired Biota Technology to help them identify the most bankable parcels. By comparing microbes in rock samples to those from oil produced in the area, Biota can map out choice draining spots and, according to founder Ajay Kshatriya, boost a well’s output by millions of dollars.

“It’s a whole new data source for an industry that’s never looked at this before,” said Kshatriya, an engineer who used to work in biotech. “New technology innovation is now existential. It’s not a nice-to-have to create a couple more points of profit margin. It’s, ‘If we don’t get more economic, our field is out of the money. So we need that technology to remain competitive.’ ”

EOG Resources Inc., an Enron Corp. castoff that is now the second largest independent U.S. oil explorer, has embraced big data analysis to such a degree that Wolfe Research analyst Paul Sankey called it “the best oil company we have ever covered.” The Houston-based company has, among other things, invented proprietary iPhone apps that field crews use to calculate how hard to frack particular stretches of crude-soaked rocks.

Autonomous Operations

The industry is years away from fully embracing autonomous operations, said Binu Mathew, who was recruited seven months ago from Oracle Corp. to be the global head of digital for General Electric Co.’s oil and gas unit and works out of GE’s San Ramon, California, research center. The company is planning to roll out a new software suite in the third quarter that will pull in data from wells connected to the cloud.

There’s a lot of promise. A truly digital oilfield could lead to a 20% cut in primary preventative maintenance costs and a 2.5% boost to production volumes, according to GE estimates. With oil production in the U.S. is at 10 million barrels a day, that could mean a 231,000 barrel-a-day boost.

As it is, only 3 to 5% of all oil and gas assets — including wells themselves and all the various equipment used to drill and frack and maintain production — are connected digitally, according to GE. And of data collected, 97% is never used.

That seems to be on track to change fast. Over the next three to five years, 50% of oil and gas companies surveyed by Accenture Plc, a said they plan to boost spending on digital technologies. Investments are expected to focus increasingly on big data and mobile devices for roustabouts.

Disrupted Industry

As Mathew sees it, the oil industry is on its way to being disrupted, to use the Silicon Valley vernacular, in the way the auto industry has been. Self-driving cars are being tested around the country and could be on the market within a decade, and even the vehicles most Americans drive today are as dependent on processors as bolts and gears to function.

A $10 million oil well should have the same round-the-clock connectivity and self-help diagnostic capabilities as a $30,000 Chevrolet. To get to that point, energy companies “have to agree to work differently,” said Schlumberger’s Belani.

Schumberger planted its flag when it opened its Technology Innovation Center in 2015 less than a mile west of Stanford University. Oil-industry veterans and newly minted tech geeks work together in the two-story building that dutifully uphold Silicon Valley traditions, with a ping-pong table, bean-bag chairs, catered lunches and yoga classes.

“It’s really trying to bring that oilfield expertise and insert it into the mix of a Silicon Valley, digital-technology way of working," said Jan Smits, an engineer who is Schlumberger’s head of software technology. “That’s where the magic happens.”

Bloomberg News by David Wethe