Danny Goodwin reports via Search Engine Land: A trove of leaked Google documents has given us an unprecedented look inside Google Search and revealed some of the most important elements Google uses to rank content. Thousands of documents, which appear to come from Google's internal Content API Warehouse, were released March 13 on Github by an automated bot called yoshi-code-bot. These documents were shared with Rand Fishkin, SparkToro co-founder, earlier this month.

What's inside. Here's what we know about the internal documents, thanks to Fishkin and [Michael King, iPullRank CEO]:

Current: The documentation indicates this information is accurate as of March.
Ranking features: 2,596 modules are represented in the API documentation with 14,014 attributes.
Weighting: The documents did not specify how any of the ranking features are weighted -- just that they exist.
Twiddlers: These are re-ranking functions that "can adjust the information retrieval score of a document or change the ranking of a document," according to King.
Demotions: Content can be demoted for a variety of reasons, such as: a link doesn't match the target site; SERP signals indicate user dissatisfaction; Product reviews; Location; Exact match domains; and/or Porn.
Change history: Google apparently keeps a copy of every version of every page it has ever indexed. Meaning, Google can "remember" every change ever made to a page. However, Google only uses the last 20 changes of a URL when analyzing links.

Other interesting findings. According to Google's internal documents:

Freshness matters -- Google looks at dates in the byline (bylineDate), URL (syntacticDate) and on-page content (semanticDate).
To determine whether a document is or isn't a core topic of the website, Google vectorizes pages and sites, then compares the page embeddings (siteRadius) to the site embeddings (siteFocusScore).
Google stores domain registration information (RegistrationInfo).
Page titles still matter. Google has a feature called titlematchScore that is believed to measure how well a page title matches a query.
Google measures the average weighted font size of terms in documents (avgTermWeight) and anchor text. What does it all mean? According to King: "[Y]ou need to drive more successful clicks using a broader set of queries and earn more link diversity if you want to continue to rank. Conceptually, it makes sense because a very strong piece of content will do that. A focus on driving more qualified traffic to a better user experience will send signals to Google that your page deserves to rank." [...] Fishkin added: "If there was one universal piece of advice I had for marketers seeking to broadly improve their organic search rankings and traffic, it would be: 'Build a notable, popular, well-recognized brand in your space, outside of Google search.'"

An anonymous reader quotes a report from Ars Technica: Some of the most infamous so-called shadow libraries have increasingly faced legal pressure to either stop pirating books or risk being shut down or driven to the dark web. Among the biggest targets are Z-Library, which the US Department of Justice has charged with criminal copyright infringement, and Library Genesis (Libgen), which was sued by textbook publishers last fall for allegedly distributing digital copies of copyrighted works "on a massive scale in willful violation" of copyright laws. But now these shadow libraries and others accused of spurning copyrights have seemingly found an unlikely defender in Nvidia, the AI chipmaker among those profiting most from the recent AI boom.

Nvidia seemed to defend the shadow libraries as a valid source of information online when responding to a lawsuit from book authors over the list of data repositories that were scraped to create the Books3 dataset used to train Nvidia's AI platform NeMo. That list includes some of the most "notorious" shadow libraries -- Bibliotik, Z-Library (Z-Lib), Libgen, Sci-Hub, and Anna's Archive, authors argued. However, Nvidia hopes to invalidate authors' copyright claims partly by denying that any of these controversial websites should even be considered shadow libraries.

"Nvidia denies the characterization of the listed data repositories as 'shadow libraries' and denies that hosting data in or distributing data from the data repositories necessarily violates the US Copyright Act," Nvidia's court filing said. The chipmaker did not go into further detail to define what counts as a shadow library or what potentially absolves these controversial sites from key copyright concerns raised by various ongoing lawsuits. Instead, Nvidia kept its response brief while also curtly disputing authors' petition for class-action status and defending its AI training methods as fair use. "Nvidia denies that it has improperly used or copied the alleged works," the court filing said, arguing that "training is a highly transformative process that may include adjusting numerical parameters including 'weights,' and that outputs of an LLM may be based, at least in part, on such 'weights.'" "Nvidia's argument likely depends on the court agreeing that AI models ingesting published works in order to transform those works into weights governing AI outputs is fair use," notes Ars. "However, authors have argued that 'these weights are entirely and uniquely derived from the protected expression in the training dataset' that has been copied without getting authors' consent or providing authors with compensation."

"Authors suing Nvidia have taken the next step, linking the chipmaker to shadow libraries by arguing that 'these shadow libraries have long been of interest to the AI-training community because they host and distribute vast quantities of unlicensed copyrighted material. For that reason, these shadow libraries also violate the US Copyright Act.'"

Nashville-based Yoshi Mobility launched in 2015 to deliver gasoline to vehicle owners, reports Forbes. But this week the company announced they'll begin converting GM electric delivery vans into "mobile EV superchargers" — fast, battery-powered 240 kw DC chargers — for corporate fleets of electric cars. "There's kind of this critical grid problem and so we think that we can accelerate towards an EV future and this is a unique way that we can do it," said [cofounder/CEO Bryan] Frist in an interview. "The mobile charger can charge and then it can multiplex all the spots. So we tell people, it can electrify every spot in your parking lot."

Each mobile supercharger can service between five and seven vehicles according to Frist. With perhaps two superchargers operating on a fleet operator's lot, one would service a vehicle while the other supercharger would replenish its own charge off the grid and they would alternate, according to Frist. "What we say is we can do that same charge instead of three and a half hours, we can do in 10 minutes, and we can move around your lot," Frist says. "You don't have to put in all the infrastructure. You don't have to build it out. You just contract with us."

The company plans to begin with a "handful" of mobile superchargers in [GM's] BrightDrop vans but expects to ramp up production and begin commercializing more widely during the first quarter of 2025... The mobile superchargers will complement Yoshi Mobility's existing offering of high-capacity mobile generators that sit on a fleet operator's lot putting out as much as a megawatt of power and can service a larger number of vehicles than the mobile units.

Thursday long-time Slashdot reader mschaffer reported that "Microsoft's search engine isn't working correctly, and many alternative search engines that rely on it are down, too."

Bing started "having issues" around 1:30 a.m. EST, reports SearchEngineLand (citing Downdector.com, and sharing screenshots of Bing.com searches failing — even on partner sites like DuckDuckGo).

By Thursday morning search capabilities for ChatGPT, Copilot, DuckDuckGo, and other platforms had stopped working, reports the Verge, saying the issues "appeared to be linked to Bing's API and any service that relies upon it." While Microsoft's own web search engine, Bing, was also seemingly affected, according to TechCrunch, it came back online eventually. By 11AM ET, OpenAI posted a note indicating the issue had been resolved, saying, "Between around 10:10 PM PT yesterday and 6:50 AM PT today, we experienced a partial outage affecting ChatGPT's web-browsing capabilities due to Bing being unavailable." DuckDuckGo posted that "we're coming back up" at around 10:30AM ET, and so did Ecosia, which is "the search engine that plants trees."
Copilot users experienced "a loading loop that prevented users from accessing the service," according to the article, while ChatGPT users attempting a web search got error messages instead.

Ars Technica adds that it also stopped searches from Microsoft's Edge browsers (that hadn't changed their default search settings). But they also had a disturbing observation for people worried that web search is dominated by Google: "most of your other major options were brought down by a single API outage... The overwhelming majority of search tools offering an alternative" to Google are using Google, Bing, or Yandex... Yandex, being based in Russia, is a non-starter for many people around the world at the moment."
But their article digs deep into the alternatives, starting with this list compiled by undergraduate CS major Rohan Kumar of search sites with their own indexes — including Mojeek, Stract, and Right Dao and Yep...

"Apple's done a fantastic job of really innovating on the Mac," Microsoft CEO Satya Nadella told the Wall Street Journal in a video interview this week.

. Then he said "We are gonna outperform them" with the upcoming Copilot+ laptops from Acer, ASUS, Dell, HP, Lenovo and Samsung that have been completely reengineered for AI — and begin shipping in less than four weeks. Satya Nadella: Qualcomm's got a new [ARM Snapdragon X] processor, which we've optimized Windows for. The battery lab, I've been using it now — I mean, it's 22 hours of continuous video playback... [Apple also uses ARM chips in its MacBooks]. We finally feel we have a very competitive product between Surface Pro and the Surface laptops. We have essentially the best specs when it comes to ARM-based silicon and performance or the NPU performance.

WSJ: Microsoft says the Surfaces are 58% faster than the MacBook Air with M3, and has 20% longer battery life.
The video includes a demonstration of local live translation powered by "small language models" stored on the device. ("It can translate live video calls or in-person conversations from 44 different languages into English. And it's fast.")

And in an accompanying article, the Journal's reporter also tested out the AI-powered image generator coming to Microsoft Paint.

As a longtime MS Paint stick-figure and box-house artist, I was delighted by this new tool. I typed in a prompt: "A Windows XP wallpaper with a mountain and sky." Then, as I started drawing, an AI image appeared in a new canvas alongside mine. When I changed a color in my sketch, it changed a color in the generated image. Microsoft says it still sends the prompt to the cloud to ensure content safety.
Privacy was also touched on. Discussing the AI-powered "Recall" search functionality, the Journal's reporter notes that users can stop it from taking screenshots of certain web sites or apps, or turn it off entirely... But they point out "There could be this reaction from some people that this is pretty creepy. Microsoft is taking screenshots of everything I do."

Nadella reminds them that "it's all being done locally, right...? That's the promise... That's one of the reasons why Recall works as a magical thing: because I can trust it, that it is on my computer."

Copilot will be powered by OpenAI's new GPT-4o, the Journal notes — before showing Satya Nadella saying "It's kind of like a new browser effectively." Satya Nadella: So, it's right there. It sees the screen, it sees the world, it hears you. And so, it's kind of like that personal agent that's always there that you want to talk to. You can interrupt it. It can interrupt you.
Nadella says though the laptop is optimized for Copilot, that's just the beginning, and "I fully expect Copilot to be everywhere" — along with its innovatively individualized "personal agent" interface. "It's gonna be ambient.... It'll go on the phone, right? I'll use it on WhatsApp. I'll use it on any other messaging platform. It'll be on speakers everywhere." Nadella says combining GPT-40 with Copilot's interface is "the type of magic that we wanna bring — first to Windows and everywhere else... The future I see is a computer that understands me versus a computer that I have to understand.

The interview ends when the reporter holds up the result — their own homegrown rendition of Windows XP's default background image "Bliss."

China's rapid solar power expansion is slowing due to grid bottlenecks, market reforms, and diminishing rooftop space, with new solar builds dropping 32% in March year-on-year. Reuters reports: The country's solar power expansion is slowing due to tighter curbs on supplying excess power from rooftop solar into the grid and changes in electricity pricing that are denting the economics of new solar projects. Forecasts show China's solar build this year will be heavily outpaced by growth in its photovoltaic (PV) module manufacturing capacity, raising the prospect the country will export more solar panels despite a trade backlash in Europe and the U.S. The main factor slowing the expansion of distributed solar - installations built near the point of use, mostly on rooftops - is that there is not enough storage or transmission capacity to soak up the excess power generated when the sun is shining. That in turn is leading regulators to take away some of the price support that led to the rapid growth of distributed solar. "In the next couple of years, this is going to be a huge problem that all provinces will face as grids are oversaturated, the infrastructure is overwhelmed," said Cosimo Ries, an analyst with Trivium China, a policy research group. [...]

Renewable generators previously enjoyed a guarantee that grid operators would buy nearly all of their power at a rate tied to the coal index. That guarantee was lifted on April 1 and took effect earlier in some places, three industry experts said. Now, renewable generation is increasingly subject to less favourable market pricing. Shenhua Energy, a state-run coal and power firm, said in its first-quarter report that prices for its solar power fell 34.2% year-on-year to 283 yuan per megawatt-hour (MWh), while its coal power prices fell just 2.4% to 406 yuan per MWh. Wang Xiuqiang, a researcher at consultancy Beijing Linghang, attributed the lower solar prices and profitability to a higher proportion of market-based pricing. At the same time, grid companies are dialling back the 5% curtailment limit, "creating the risk for project owners that their generation might not be bought", said David Fishman of Shanghai-based energy consultancy the Lantau Group.

Curtailment for Huaneng Power International, a major state-owned generator, rose to 7.7% in the first quarter from 3.1% a year earlier, Jefferies analysts said in a client note, citing Huaneng management. In a further challenge, the easiest-to-site projects have already been largely developed, said Shi Lida, research manager at Yongan Guofu Asset Management. At sites still available, rooftops may need to be reinforced, grid connections may be limited, or hours of sunlight may be short. "If your costs don't continue to fall, the investment will not be cost effective," Shi said. Further reading: Germany Has Too Many Solar Panels, and It's Pushed Energy Prices Negative

Scientists report that Greater Spotted Eagles altered their migration routes across Ukraine to avoid conflict and habitat destruction caused by the war. The BBC reports: The scientists studied GPS data from tagged birds in the months after the February 2022 invasion, a time of heavy fighting in northern Ukraine as Russia tried to take Kyiv by sending troops south from Belarus. The researchers from the Estonian University of Life Sciences and the British Trust for Ornithology reported their findings in the journal Current Biology. "The war in Ukraine has had a devastating impact on people and the environment. Our findings provide a rare window into how conflicts affect wildlife," said lead author Charlie Russell, a postgraduate researcher at the University of East Anglia.

Classified as a vulnerable species, the Greater Spotted Eagle is a large, brownish-colored bird of prey. Researchers started following them using GPS tracking devices in 2017 but didn't expect to be monitoring them through an active conflict zone five years later. The findings reveal they made large deviations from their previously tracked routes. They also spent less time stopping at their usual refueling sites in Ukraine or avoided them entirely. As a result, they traveled farther, about an extra 52 miles (85km) on average.

For migrating birds, stopover sites are essential places to get food, water, and shelter. These changes delayed the birds' arrival at the breeding grounds and likely made them use more energy, to damaging effect. "No doubt about it. I think the take-home story is that the conflict in Ukraine is fundamentally disrupting the migratory ecology of this species," said Dr Jim Reynolds, Assistant Professor in Ornithology and Animal Conservation at the University of Birmingham, who was independent from the study. "For a vulnerable species like this, anything that disrupts breeding performance is a major problem. As a conservation biologist, you worry about that in a massive way." Despite all the tagged birds surviving, researchers believe their experience may have affected their ability to breed.

California is another state lining up to pass a law requiring adult sites to verify the ages of porn watchers. From a report: The California State Assembly passed the Parent's Accountability and Child Protection Act that will require porn companies doing business in the state to verify that users are 18 years or older. This law would also affect other businesses such as fireworks, body branding, and even BB guns. Democrat Rebecca Bauer-Kahan and Republican Juan Alanis pushed for passage of the bill, which ended up receiving 65 out of possible 80 yes votes, and zero no votes with 15 assembly members listed as not voting. Before the bill becomes law, it still has to pass the State Senate and then be signed by Governor Gavin Newsom. Louisiana was the first state to pass an age verification law for adult sites in 2022. In the past year, several other states jumped on the bandwagon including Utah, Arkansas, Florida, Indiana, Mississippi, Montana, North Carolina, Texas, and Virginia.

Microsoft's Windows Recall feature is attracting controversy before even venturing out of preview. From a report: The principle is simple. Windows takes a snapshot of a user's active screen every few seconds and dumps it to disk. The user can then scroll through the snapshots and, when something is selected, the user is given options to interact with the content.

Mozilla's Chief Product Officer, Steve Teixeira, told The Register: "Mozilla is concerned about Windows Recall. From a browser perspective, some data should be saved, and some shouldn't. Recall stores not just browser history, but also data that users type into the browser with only very coarse control over what gets stored. While the data is stored in encrypted format, this stored data represents a new vector of attack for cybercriminals and a new privacy worry for shared computers.

"Microsoft is also once again playing gatekeeper and picking which browsers get to win and lose on Windows -- favoring, of course, Microsoft Edge. Microsoft's Edge allows users to block specific websites and private browsing activity from being seen by Recall. Other Chromium-based browsers can filter out private browsing activity but lose the ability to block sensitive websites (such as financial sites) from Recall. "Right now, there's no documentation on how a non-Chromium based, third-party browser, such as Firefox, can protect user privacy from Recall. Microsoft did not engage our cooperation on Recall, but we would have loved for that to be the case, which would have enabled us to partner on giving users true agency over their privacy, regardless of the browser they choose."

It seems no industry is safe from consolidation, and the latest target is gaming media. From a report: IGN Entertainment has acquired the website portfolio of UK publisher Gamer Network, which operates a number of beloved games-focused publications. That list includes Gamesindustry.biz, Eurogamer, Rock Paper Shotgun, VG247, and the tabletop site Dicebreaker. The network also holds shares in sites like Nintendo Life and Digital Foundry.

Terms of the deal were not disclosed. Gamesindustry.biz reports that "some redundancies" have been made across the sites, though it's not clear how many workers have been impacted. According to several posts on X, editors at both Rock Paper Shotgun and Gamesindustry.biz have been laid off. IGN Entertainment is owned by Ziff Davis, which, in addition to IGN's site, also operates other subsidiaries like Humble Bundle.

Microsoft is planning to either add subtitles or even dub video produced by major video sites, using AI to translate the audio into foreign languages within Microsoft Edge in real time. From a report: At its Microsoft Build developer conference, Microsoft named several sites that would benefit from the new real-time translation capabilities within Edge, including Reuters, CNBC News, Bloomberg, and Coursera, plus Microsoft's own LinkedIn. Interestingly, Microsoft also named Google's YouTube as a beneficiary of the translation capabilities. Microsoft plans to translate the video from Spanish to English and from English to German, Hindi, Italian, Russian, and Spanish. There are plans to add additional languages and video platforms in the future, Microsoft said.

The U.S. Environmental Protection Agency on Monday urged water utilities to take action to improve their digital defenses, following a spate of recent cyberattacks. From a report: The agency's "enforcement alert" said that recent inspections of water systems found that more than 70 percent fail to meet basic cybersecurity standards, including some with "critical" vulnerabilities, such as relying on default passwords that haven't been updated and single logins that "can easily be compromised." The notice comes after a Russian hacktivist group claimed credit for digital assaults on water sites in Texas and Indiana. Late last year, Iran-linked Cyber Av3ngers group took responsibility for striking a water authority in Pennsylvania.

A new Pew Research Center analysis shows just how fleeting online content actually is: 1. A quarter of all webpages that existed at one point between 2013 and 2023 are no longer accessible, as of October 2023. In most cases, this is because an individual page was deleted or removed on an otherwise functional website.
2. For older content, this trend is even starker. Some 38% of webpages that existed in 2013 are not available today, compared with 8% of pages that existed in 2023.

This "digital decay" occurs in many different online spaces. We examined the links that appear on government and news websites, as well as in the "References" section of Wikipedia pages as of spring 2023. This analysis found that:
1. 23% of news webpages contain at least one broken link, as do 21% of webpages from government sites. News sites with a high level of site traffic and those with less are about equally likely to contain broken links. Local-level government webpages (those belonging to city governments) are especially likely to have broken links.
2. 54% of Wikipedia pages contain at least one link in their "References" section that points to a page that no longer exists.[...]

An anonymous reader quotes a report from Ars Technica: It's amazing, and a little sad, to think that something created in 1989 that changed how people used and viewed the then-nascent Internet had nearly vanished by 2024. Nearly, that is, because the dogged researchers and enthusiasts at The Serial Port channel on YouTube have found what is likely the last existing copy of Archie. Archie, first crafted by Alan Emtage while a student at McGill University in Montreal, Quebec, allowed for the searching of various "anonymous" FTP servers around what was then a very small web of universities, researchers, and government and military nodes. It was groundbreaking; it was the first echo of the "anything, anywhere" Internet to come. And when The Serial Port went looking, it very much did not exist.

While Archie would eventually be supplanted by Gopher, web portals, and search engines, it remains a useful way to index FTP sites and certainly should be preserved. The Serial Port did this, and the road to get there is remarkable and intriguing. You are best off watching the video of their rescue, along with its explanatory preamble. But I present here some notable bits of the tale, perhaps to tempt you into digging further.

An anonymous reader quotes a report from UploadVR: 1 in 4 U.S. teens told Pew Research Center they play games on a VR headset. The survey was conducted on 1453 U.S. teens aged 13 to 17. Pew claims the participants were "recruited primarily through national, random sampling of residential addresses" and "weighted to be representative of U.S. teens ages 13 to 17 who live with their parents by age, gender, race and ethnicity, household income, and other categories." Broken out by gender, 32% of boys and 15% of girls said they play games on a VR headset. The survey doesn't ask whether they actually own the headset, so this will include those who play on a sibling or parent's headset.

samleecole writes: For the last decade, farmers have been warning that John Deere, a company celebrated by farmers, country musicians, and politicians, has been doing something else very American: Concentrating power, stripping away the ownership rights of people who buy their products, and adding a bevy of artificial, software-based repair restrictions that have effectively created a regime in which farmers can no longer fix their own tractors, combines, harvesters, and other agricultural equipment. Farmers have resorted to pirating John Deere's software and firmware on underground forums and torrent sites, and have used software cracked by Ukrainian pirates in order to simply fix the things they own. Farmers often have to wait days or weeks for an "authorized" John Deere dealership to come to their farms to repair their equipment, meanwhile their crops die on the vine.

For years, very little happened to slow down John Deere's march toward total control of the repair market. But interviews with farmers, activists, and lawyers, and a review of court records reveal a turn in the story: There is increased scrutiny on Deere's repair practices not just in this class action lawsuit, but from state legislators, the White House, and a series of federal agencies. The walls on Deere's repair monopoly may finally be closing in.