Google Stadia

Haha.. I dunno. I think I've made peace with the fact that I'll never play all the games I've purchased. I'm going to damn well try to play them instead of buying new ones though.

 

My Nexus 6P started turning off at 30% battery after having it for about a year or so. They gave me a full refund of my original purchase price (always have always bought my nexus/pixel phones through them) and I bought a Pixel XL instead. My wife used her Pixel protection and they cross shipped her a replacement phone no questions asked. I haven't had very bad experience with their CS yet, though I know others have (just like pretty much any company).

This Stadia launch seems like something special though. They've managed to pull this off somewhat badly.

That black friday deal on the Pixel 3a was too good to pass up, we just bought one of each of those. My Pixel XL's battery is losing health after so many years of charge cycles, and the Pixel 4 XL i just too expensive even at $200 off.

 

You're a very lucky person if you had good experience with Google tech support. If you will look at Google Pixel subreddit, you will find plenty of threads about "Google has the worst customer support". Threads such as
https://www.reddit.com/r/GooglePixel/comments/dwu4fr/is_googles_customer_service_just_terrible_now

You will not find so many threads with large amount of replies about customer support in subreddits related to Samsung smartphones or Apple phones. Now take a look at the sales charts and see how many smartphones Samsung and Apple sells and how many Google sells ;-) And yes, there are other companies with bad customer support but none of them sells flagship device for $900 in US (not counting temporary sale that is going on right now for Pixel devices). A flagship device with multiple unfixable flaws due to incompetent engineers and incompetent management, such as inability to take 4k60fps videos, inability to have more than 128GB of storage, flawed physical build quality:

And flawed material quality:
https://www.reddit.com/r/GooglePixel/comments/dyt4cj/more_fuel_for_pixel_4_buyers_remorse_coating_on

So whether someone was lucky with their Google devices and customer support - there are much more who were not (compared to other brands) and with issues which are much more severe and inexcusable for such expensive products. This is the reason I did not want to gamble anymore by buying more Pixel devices (after returning second Pixel 3a XL with defective display and vibration motor) and why I did not expect the Stadia to launch smoothly and I do not believe this service will be still available after a couple of years.

 

Yes, I know Sony has their own streaming service, unfortunately it is also flawed - it is affected by same issues with latency and video bitrate which may randomly decrease due to various reasons (congestion on particular routing node or ISP throttling you) making whole game more blurry. I think this is the reason Sony does not actively advertise it even though it has more benefits than Stadia such as lower price and larger amount of games and this is why they will still be producing PS5 with optical drive.

 

Those issues can happen with any game streaming service. I did the 7-day trial of PS Now and had good luck with it. Next time Sony runs a sale on a subscription I might sign up.

 

googles biggest mistake was claiming it could replace a ps4 or xo. we are at least 3 years away from a streaming service offering no hiccups in IQ and latency. I currently use geforce now and while not quite as good as stadia I don't mind it for witcher 3 or project cars 2. I get random lag spikes like once every minute and I simply put up with it and appreciate the fact im playing on a weak computer. stadia might have flopped but this still is the future unless laptops desktops or consoles offer photo realistic graphics this next coming gen as that will be it and we will never require upgrades in terms of graphics horsepower..pending game developers

 

https://www.polygon.com/2019/12/5/20997365/google-stadia-darksiders-genesis-price-steam-pro-discount

LOLWUT

 

Promises made, promises broken.

 

Some good Stadia news for a change...?

Metro Exodus on Stadia Tested! Better than Xbox One X?
Dec 11, 2019
Digital Foundry
A proper 4K experience is in the bag for Stadia - which is great. Developer 4A's work here puts it in the domain of console visuals rather than PC's otherwise - and so the question is, is it enough to overtake Xbox One X?

 

Womp, womp.



https://store.gamersnexus.net/products/disappointment-pc-t-shirt-2019-cotton-black

 

Workaround for running Stadia on non-Pixel Android phones...

Google Stadia on any android phone! No root, full wireless controller functionality
Dec 18, 2019
Modern Family Life and Travel
Google Stadia running on my Oppo 10x Zoom, no root needed... Quick demo of Stadia running on a non Pixel non Google Android phone. I'm sure google will get round to making Stadia function as promised, but for those early adopters like me, here's how you can sample Stadia on any Android device
Link to Info: https://piunikaweb.com/2019/11/29/pla...
APK for controller functionality: https://github.com/sigmaxipi/chromium...

 

More money, latency, and lies. Less value, stability, and availability.



Speaking of latency, here's some science testing Google's claims.

https://fdossena.com/?p=stadialies/i.md

 

Im using geforce now and getting 35ms latencty and its amazing how the heck can stadia be this bad I call BS

 

Eventually they get there...

You can buy the Stadia Controller Claw from Power Support right now
Abner Li, Dec. 18th 2019 3:36 pm PT
https://9to5google.com/2019/12/18/buy-stadia-controller-claw/
https://www.powersupportintl.com/power-support-claw-for-stadia-controller/

Stadia has Collectors Editions of Major releases, just like PC / Consoles...

$380 Darksiders Genesis "Nephilim" Collector's Edition Unboxing (Google Stadia Gameplay)
Dec 7, 2019
TheRelaxingEnd
Unboxing Darksiders Genesis Nephilim Limited Collector's Edition (a board game included). 5000 units made. Google Stadia console gameplay test. Thanks to THQ Nordic for sending this box for unboxing!


Hmmm, now that's an unboxing channel

 

I don't know what to say I run games off my own gpu and get stutters so im not sure what people are talking about when they say stadia stutters...is it bad or really bad stadia owners chime in

 

Windows Central compiled a list of games vs resolution vs FPS performance, and Xbox One X came in on top. Stadia has a long ways to go - with hardware upgrades next year + roll out of the free plan + software porting tuning will hopefully bring a continual wave of improvements (pun intended).

Google Stadia vs. Xbox One X game resolution and frame rate comparison
Xbox One X comes out on top.
ASHER MADAN, 18 Dec 2019
https://www.windowscentral.com/google-stadia-vs-xbox-one-x-resolution-and-frame-rate-comparison

 

It's almost impressive how Google continues to screw this up.

 

It's really not that unusual to have a ported game come out with the version they started porting originally - rather than incorporating the code changed after they got their working copy.

It creates too much uncertainty - will the changes actually work or will the bugs brought out with a new revision create more problems than deliver new features.

Better to work with a known quantity, release that, then integrate the updates as quickly as possible with feedback on the other platforms as to what does and doesn't work - and roll in those fixes before releasing the new version to sync up with the other platforms.

If they work it right they may release the fixed revision DLC before the other platforms roll out their fixes.

It's not always going to happen this way - when new games come out on all platforms simultaneously the version will be the same across all platforms.

Stadia is playing catch up being the new kid on the block, what with BL3 releasing just weeks before Stadia. There were likely staffing resources issues at Gearbox which kept Stadia from getting liaison and source for the new updates as normal from the original developers.

Gearbox and stores are are busy with DLC bugfixes rollout ahead of Stadia, and wouldn't have time to simultaneously work with Google to drop their Stadia DLC / version updated release as Stadia's first release.

Yongyea isn't a software developer so he wouldn't know about these release issues, how the companies need to prioritize and synchronize their resources and how new services come up last in the queue often until time goes by and the relationship builds.

It takes years to get these things right, and often there are always such issues when coordinating releases to many different customers - especially new customers. Stadia will continue to push forward and smooth such things out, or they won't, every service is different and Stadia needs to work through their start up teething problems.

 

Given Stadia's screw-ups up to this point, their lack of willingness to seem to fix those screw-ups correctly, and Google's perpetual mindset of "it's a beta product" throughout their software suite, I wouldn't put money on Stadia lasting for very long and thus becoming OnLive v2. Especially given the inherent issues of all-online products like this.

 

PlayStation Now and GeForce Now had smoother launches, work far better, and are cheaper to use. This is some Grade-A cope right here.

 

Stadia has been out what? All of 4 weeks? How are they supposed to fix everything in 4 weeks? Get real. It's the Holiday!!

We'll have to give Stadia 6 months to a year to see how much progress is made. Games take 18-24 months you know, and Rome wasn't built in a day, the actual time it takes to do things is far longer than you can know.

It seems you're willing to give up even quicker than Google. What's up with that?

Googles got plenty of money to keep improving things. Let's give them the time to get things spruced up. Maybe there will be free tier with complementary games as demo to play some day.

Hey, it could happen.

 

Fix the easy issues before releasing the product? Such as not actually delivering codes for the people who spent triple-digit prices for the device early. And actually QC test the product with home internet connections to see if things actually work as promised, such as minimal lag and jitter in-game (but they don't, but let's ignore the fundamental issues with this sort of business).

I'm "giving up" on all-online gaming services like OnLive and Stadia because it's a business model that doesn't really work, especially for games where input timing is critical (such as a FPS game). It's just a fundamental issue with the tech, plus the cost of the tech for the end user making it financially silly compared to just buying/building a gaming laptop or desktop or even a console ( especially a computer). Why spend (in this case) $130 + a monthly fee (or just a monthly fee later when Google releases Stadia to the poors) when you can build a half-decent gaming desktop (or buy a console) for somewhere around $500-$1000 all-in or a budget gaming laptop for ~$1000 all-in, especially when a PC/console is going to actually play the games well vs the numerous technical (and business) issues with OnLive/Stadia/whatever.

The whole idea itself "gives up" on itself, no need to berate the people calling out its future failure

Furthermore, releasing half-a**ed products and expecting your brand-name clout to carry you through is how you get stuff like Fallout 76 (and Stadia).

 

Stadia does not include any games. You must pay for those as well, with many titles commanding the $60 price tag they had when they were first released.

Right now, you can buy the disc-less Xbox One S for $150 brand new. A couple weeks ago, you could have picked up a refurbished OG Xbox One with a one-year warranty for $115. Microsoft, Sony, and Steam are all offering sales on games right now so you can grab a nice library to go along with your new console for less than $300. If your Internet connection is good enough for Stadia, it's good enough for digital downloads.

 

Let's continue after the new year. Happy Holidays

 

FYI, Comcast let us add $10/mo unlimited data with no time limit - no 3/6/12 month limit. Normal price is $50/mo.

IDK if this is the new norm, or an exception, but it's worth giving a call to Comcast and ask them how much is unlimited data per month? What's the best price you can give me during the Holiday?

That's all I asked, and I expected it to be a duration or time limited offer, but nope, $10/mo every month moving forward. Hopefully Comcast is well aware many people need to be to lifting the data cap for normal use moving forward.

In my case I carefully bump up to the 1TB/mo limit every month, and went over in November - using one of the banked 2 months of free overages. IDK if that helped indicate usage to the operator or not and allowed him to offer the lower price or not.

Good luck, and Happy New Year

 

It's getting so we can't have new innovation without everyone wanting to jump on the bandwagon to kill it before it has time to mature.

It's a very sad commentary on how people can use one or two things that they think they "know" so they kill it before it even has a chance to grow. They can't understand how things improve and change if given time.

After seeing so much technology not meet it's goals when it first comes out, and how many iterations are needed to get to another level of success, I can honestly say that Google Stadia isn't the worst I've seen.

Especially as Google has to herd a bunch of hardware, network, software, and 3rd party software developers in sync to get everything out the door. Every one of those points of innovation need to be given time to mature.

On The Fence: Should You Subscribe To Google Stadia?
Jan 1, 2020
IGN
Still on the fence about whether to subscribe to Google Stadia? As with any new tech launch, there are plenty of heated opinions on both sides. Some call it the 4k streaming solution that will pave the way for our collective gaming future. Others think it's bad. Brian Altano and Max Scoville dig into the reasoning on both sides, all in a kindhearted effort to help Michael Swaim and steal things from his home for later resale.


The comments are as expected. And if anything the failure to imagine the future possibilities for Stadia if given a chance to grow will be what kills Stadia.

Don't blame Google if they decide to end Stadia prematurely due ingratitude among the gaming community for Google's efforts to pioneer remote gaming technology. You are all doing it to yourselves.

 

It's not a "bandwagon" if it really is that bad.

Even ignoring the previous comments I've (and others) made, there's also the issue of the two offered resolutions (1080p or 4K) being upscaled, which kind of makes the thing even less appealing. To say nothing about the silly bs like "Stadia can predict your next input choices".

It might not be the worst you've seen, and it might even not be the worst Google product ever released maybe (Google+ says hi?). It's pretty piss poor though and I certainly can't see the appeal to it when better, non-streaming alternatives exist, especially for the money. Imagination is cute and all, but it's implementation that matters.

I'll say it again to drill the point home: Don't released half-baked products if you are expecting the public to have an overall positive view of said product. Get your ducks in a row *before* launch and don't make stupid screw-ups like leaving your "premium" customers hanging in the wind (wrt usernames and other issues previously commented on).

 

True, Google needs to learn that "normals" don't take to the interrate to innovate method of productization.

Here in Silicon Valley (and elsewhere it thrives) we are used to it and it's well understood, in fact it is embraced by most (all?) product development because we know it works.

Outside of the supportive native of those that are used to it the "normals" tend to look for any reason to kill what they don't understand.

Hopefully Google will push through this and keep going through the early criticisms and gives it a few hardware upgrades and seasons to allow Stadia to catch on.

Some in the media are still giving it a chance, and that is nice to see:

Stadia Pro adds Rise of the Tomb Raider: 20 Year Celebration and Thumper as free games (Update: Available now)
Zach Laidlaw, 9 hours ago
https://www.androidpolice.com/2020/...ation-and-thumper-as-free-games-on-january-1/

"For all the mixed press Stadia has received since launch in November, the cloud gaming service shows a ton of promise. But the real trial of Google's gaming chops will come when the free Base tier rolls out in 2020. To help the paid Pro side stand out, Google is adding two more free games to the list of perks: Rise of the Tomb Raider: 20 Year Celebration and Thumper."

" UPDATE: 2020/01/02 4:33AM PST BY SCOTT SCRIVENS
Available now

Rise of the Tomb Raider: 20 Year Celebration and Thumper are now available to Stadia Pro subscribers free of charge, so go ahead and claim them while you can. These bring the total up to six free games that have gone out to Pro tier users up to now, although only five of them are still up for grabs — you can no longer claim Tomb Raider: Definitive Edition for nothing. Check out our full list of Stadia games here."

Google Stadia: Subscription cost, games list, free games, compatibility requirements, and more (Update: January's free games)
Taylor Kerns, Dec 28, 2019
https://www.androidpolice.com/2019/...games-release-date-performance-compatibility/

"In fall 2018, Google made its interest in gaming known with Project Stream, a beta that let users play the high-end Assassin's Creed Odyssey from a humble Chrome tab. The following spring, it announced Stadia, a full-fat gaming platform that would leverage the company's computing and networking prowess to provide users access to games with no dedicated gaming hardware required. That service is available now ( in a limited capacity), and it's getting a lot of attention — both positive and negative. So what's the deal? Here's everything you need to know..."

Tomb Raider Trilogy: Stadia's Most Impressive Ports? Full Xbox One X Comparisons!
Dec 5, 2019
Digital Foundry
Join Alex Battaglia for a tour of Tomb Raider Definitive Edition, Rise of the Tomb Raider and Shadow of the Tomb Raider, streamed via Google Stadia. We're still looking at a range of Stadia ports in the background but thus far these three titles are the most impressive conversions we've seen.


Trick to INCREASE stream quality on Chrome - Force VP9 decoding
Posted by u/ColonialReddit098 9 hours ago
https://www.reddit.com/r/Stadia/comments/eiyfhg/trick_to_increase_stream_quality_on_chrome_force/

[Tampermonkey] Monitor your stream
Posted by u/AquaRegia 1 day ago
level 1 AquaRegia 1 day ago·edited 1 hour ago
"I've written another script for Tampermonkey! This one will add an overlay to your games, as you can see in the screenshot. This overlay can be moved around or hidden by pressing ctrl+m (while the game isn't fullscreen).

There's actually already a tool that (I'm guessing) works kind of like mine, it's internally called "chromeclient_devtools" but is disabled by default, so I can't use it.

EDIT: New version, now with time latency session time and average data use jitter buffer!

Okay that last one may require an explanation. For obvious reasons, you can't buffer a real-time video stream like you normally could when watching a YouTube video for example. However, some buffering is still being done, and "jitter buffer" shows how long each frame is staying in the buffer before being used."

https://imgur.com/8WdmZHw

Here's a short video showing the Stadia data overlay and it's also showing forced VP9 streaming:

60FPS TRICK TO INCREASE STREAM QUALITY FOR GOOGLE STADIA DESTINY 2 WITH VP9 DECODER
Jan 2, 2020
viral respawn
Running Google Stadia Destiny 2 on VP9 Decoders at 60 FPS. VP9 Decoder vastly improves the graphics and latency issues. Game ran like butter no issues.
Follow this REDDIT post: https://www.reddit.com/r/Stadia/comments/eiyfhg/trick_to_increase_stream_quality_on_chrome_force/


If you can keep a positive mental attitude, wonders can be done if you apply yourself to the problems, even for Google Stadia

 

Here's the thing, there are already streaming services that offer a better user experience with more games and a fairer pricing model. Once Project XCloud fully launches, assuming Microsoft has a fair pricing model and games lineup, that will be the final nail in Stadia's coffin.

 

I don't know about SV, but here in RTP (the east coast SV) we know not to release half-a**ed products (most of the time). Google better perform some major fixes to Stadia if it has any hope of surviving longer than Google's other half-baked Beta products.

 

Assuming Microsoft has a functioning brain somewhere within the organization (and I'm led to believe this), I can easily see that as being another nail as well.

 

Microsoft is conducting a public beta test of XCloud. It’s currently limited to Android devices but that’s still far more than what Google did.


Sent from my iPhone using Tapatalk

 

Google did have an internal "alpha" or "beta", but it is odd that Google didn't explicitly say the public release was a Beta - because for the longest time Google always calls every new product they release to the public for the first time a Beta.

Maybe Google just forgot to say Beta? Perhaps Google assumed everyone would know based on decades of Google releasing everything they do as a Beta.

Or after doing every Google service and product release as an iterative development Beta release for so long Google assumes that's what a normal product release is.

Google should have stuck with labeling every new product a Beta:

Google Stadia should have stayed in beta
It’s only good for people who know exactly what they’re getting into
By Dieter Bohn @backlon Dec 18, 2019, 10:07am EST
https://www.theverge.com/2019/12/18...a-editorial-assassins-creed-beta-game-library

I do recall from a while back (2010?) Google saying that they wanted to get out of the habit of releasing Beta's, but they gotta understand what a real release is to everyone else before they can accomplish that. I just laughed when they said that, and so did everyone else at the time.

That's what Google does, and if it succeeds it will be because every user pitches in and helps Google with feedback and bug reports, that's integral to the success of every Google product.

 

This is an interesting glimpse behind the scenes of the Rage 2 port:

Google Stadia Port Troubles Blamed on the Linux Kernel Scheduler
By Nathaniel Mott 15 hours ago
https://www.tomshardware.com/news/google-stadia-port-issues-linux-kernel

"... "I overheard somebody at work complaining about mysterious stalls while porting Rage 2 to Stadia. The only thing those mysterious stalls had in common was that they were all using spinlocks. I was curious about that because I happened to be the person who wrote the spinlock we were using. The problem was that there was a thread that spent several milliseconds trying to acquire a spinlock at a time when no other thread was holding the spinlock. Let me repeat that: The spinlock was free to take, yet a thread took multiple milliseconds to acquire it. In a video game, where you have to get a picture on the screen every 16ms or 33ms (depending on if you’re running at 60Hz or 30Hz), a stall that takes more than a millisecond is terrible. Especially if you’re literally stalling all threads."

Skarupke said he spent months investigating the issue before concluding that "most mutex implementations are really good, that most spinlock implementations are pretty bad and that the Linux scheduler is OK but far from ideal." He eventually decided to apply the band-aid solution of switching from a spinlock to a mutex."

I find it hard to believe Google didn't provide a guide for developers that would have called out the differences, or that Linux developers wouldn't have already known to test the difference between spinlock and mutex for their code on the host system before picking which one to use:

Spinlock vs mutex performance
"The fundamental difference between spinlock and mutex is that spinlock keeps checking the lock (busy waiting), while mutex puts threads waiting for the lock into sleep (blocked). A busy-waiting thread wastes CPU cycles, while a blocked thread does not.Oct 6, 2011"
https://attractivechaos.wordpress.com/2011/10/06/multi-threaded-programming-efficiency-of-locking/

"Here is what I have learned from this microbenchmark:

• For multi-threaded programming, it is preferred to let threads interact less with each other as CPU has to pay a price for frequent cross-talk between threads. This is true even if we do not use any lock (see lock-free vs. buffer+mutex); frequent locking/unlocking is much worse (see mutex vs. buffer+mutex). A proper design of the algorithm is more important than choosing between spinlock, mutex and semaphore.
• In this benchmark, the lock-free implementation is less efficient than buffer+mutex, but a lock-free algorithm may be more scalable to massive CPU cores because locking will become more often given more concurrent threads. When frequent locking/unlocking can hardly be avoided, a lock-free algorithm, if possible in the first place, may deliver better performance, too.
• When locking is infrequently, it does not matter too much what type of lock to use – little time goes to locking anyway. When locking is frequent, it is worthwhile to explore both spinlock and mutex. Although using mutex is usually safer and more portable, spinlock may be faster (careless micro-benchmarks tend to prefer spinlock actually). Nonetheless, let me emphasize again that we should first try our best to reduce cross-talk between threads, which is more effective than playing with different types of locks.
• The performance of locking is CPU, OS and library dependent. It is worth testing our programs on multiple platforms."

Here is Malte's blog post:

Measuring Mutexes, Spinlocks and how Bad the Linux Scheduler Really is
by Malte Skarupke
https://probablydance.com/2019/12/3...ks-and-how-bad-the-linux-scheduler-really-is/

 

https://m.slashdot.org/story/365488

TLDR: Above article about the Linux schedulers and Stadia is bollocks

 

TLDR: Linus is hard to work with.

Seeing his prose the first time can be shocking and wilting to a developer - abuse that is undeserved in most all cases, which is why Linus is "retired" from actively leading the Linux Kernel development - he's not supposed to send scathing emails to developers any longer - I guess he just can't stop.
https://otter.technology/blog/2018/09/20/linux-kernel-hastily-adopts-standard-code-of-conduct/

After Years of Abusive E-mails, the Creator of Linux Steps Aside
By Noam Cohen, September 19, 2018
https://www.newyorker.com/science/e...sive-e-mails-the-creator-of-linux-steps-aside

Here's Linus response and Malte's response, it's a long thread now:

Linus Torvalds has now responded:
"The whole post seems to be just wrong, and is measuring something completely different than what the author thinks and claims it is measuring.

First off, spinlocks can only be used if you actually know you're not being scheduled while using them. But the blog post author seems to be implementing his own spinlocks in user space with no regard for whether the lock user might be scheduled or not. And the code used for the claimed "lock not held" timing is complete garbage.

It basically reads the time before releasing the lock, and then it reads it after acquiring the lock again, and claims that the time difference is the time when no lock was held. Which is just inane and pointless and completely wrong...

[T]he code in question is pure garbage. You can't do spinlocks like that. Or rather, you very much can do them like that, and when you do that you are measuring random latencies and getting nonsensical values, because what you are measuring is "I have a lot of busywork, where all the processes are CPU-bound, and I'm measuring random points of how long the scheduler kept the process in place".

And then you write a blog-post blaming others, not understanding that it's your incorrect code that is garbage, and is giving random garbage values... [Linus T is has been wrong in this as well]

You might even see issues like "when I run this as a foreground UI process, I get different numbers than when I run it in the background as a batch process". Cool interesting numbers, aren't they?

No, they aren't cool and interesting at all, you've just created a particularly bad random number generator...

[Y]ou should never ever think that you're clever enough to write your own locking routines.. Because the likelihood is that you aren't (and by that "you" I very much include myself -- we've tweaked all the in-kernel locking over decades, and gone through the simple test-and-set to ticket locks to cacheline-efficient queuing locks, and even people who know what they are doing tend to get it wrong several times).

[We shouldn't give up improving kernel features, and Linus often hits hard and rudely like this when it's uncalled for, which is why Linus isn't running the Linux Kernel development team any longer]

There's a reason why you can find decades of academic papers on locking. Really. It's hard."

Original author of blog post responds:
"It really means a lot to me that Linus responded," the blogger wrote later, "even if the response is negative." They replied to Torvalds' 1,500-word post on the same mailing list -- and this time received a 1900-word response arguing "you did locking fundamentally wrong..."The fact is, doing your own locking is hard. You need to really understand the issues, and you need to not over-simplify your model of the world to the point where it isn't actually describing reality any more...

Dealing with reality is hard. It sometimes means that you need to make your mental model for how locking needs to work a lot more complicated..."

Here's Malte's [thread] response to Linus:

No nuances, just buggy code (was: related to Spinlock implementation and the Linux Scheduler)
By: Malte Skarupke ([email protected]), January 4, 2020 11:22am

Spoiler: Long...

Room: Moderated Discussions
Linus Torvalds ([email protected]) on January 3, 2020 6:05 pm wrote:
> The whole post seems to be just wrong, and is measuring something completely
> different than what the author thinks and claims it is measuring.
>
> First off, spinlocks can only be used if you actually know you're not being scheduled
> while using them. But the blog post author seems to be implementing his own spinlocks
> in user space with no regard for whether the lock user might be scheduled or not. And
> the code used for the claimed "lock not held" timing is complete garbage.

Hi, original author here. First of all, I'd like to point out that I actually think we mostly agree on the spinlock vs mutex discussion. My original point of this was to try to find benchmarks that show that mutexes should be preferred over spinlocks. It then accidentally turned into an investigation into the Linux scheduler, but what can I do? If the data leads that way, I have to follow. I just had to investigate giant outliers like that.

So I think we agree about the guidance on spinlock vs mutex. (so I'll skip on responding in detail to most of your text) The problem is that people use spinlocks. In video games they're quite common. Why is that? Because most simple benchmarks will show that spinlocks are better. I linked to three of those benchmarks near the top of my blog post. So it's nice to say "you shouldn't use spinlocks" but good programmers will verify those claims, and if their benchmarks show otherwise, they'll use spinlocks. Before my blog post I had a very hard time finding guidance that actually backed up the claim "you shouldn't use spinlocks" with benchmarks. Part of my goal was to provide that.

I think the part where we disagree is in whether I showed that the Linux scheduler is problematic or not. Is this a valid benchmark?

> So now you still hold the lock, but you got scheduled away from the CPU, because you had
> used up your time slice. The "current time" you read is basically now stale, and has nothing
> to do with the (future) time when you are actually going to release the lock.

Right so this is a hard thing to measure, but this is the best thing I could come up with, so give me a chance to defend it: There are three cases where a large amount of time can pass between the "end time" and the "start time":
1. if the lock actually sits idle and nobody takes it.
2. (the one you're mentioning) the thread holding the lock takes a timestamp and then doesn't actually unlock the lock before being scheduled away.
3. a thread takes the lock and gets scheduled away before taking the initial timestamp.

To reason about whether this distorts the measurements or not, let's talk about the ticket_spinlock, because in that one the behavior is exactly defined. There is potentially a thread holding the lock, (let's call it C for "current") and there is always one thread who is next in line. (let's call it N for "next") All the other threads will call _mm_pause() and yield() in a loop.

I think we both agree that case 1 was the thing I was trying to measure so if that happens we're fine and the measurements are correct. In that case the scheduler for some reason decides to not run thread N even though all other threads are calling yield().

In case 2 thread C is not being run by the scheduler. All fifteen other threads are calling yield(). Thread N is also calling yield(), and does not look special in this situation.

Case 3 is the same situation, thread C is still not being run even though fifteen other threads are calling yield().

Once we break it down like that we realize that actually these are all the same case. In all of these cases one thread can run, all others are calling yield(). The only difference between the case that I wanted to measure and the other two accidental cases is whether the scheduler is incorrectly not running thread C or incorrectly not running thread N. In either case all other fifteen threads are just calling yield().

So your claim is that it's a problem that I try to measure how long it takes for thread N to run even though I might accidentally be measuring how long it takes for thread C to run. But I claim that that's fine because these are all equally problematic. One thread could run, all other threads are yielding, yet that one thread is not running. And we don't care whether the thread that could run is thread N or thread C.

Of course I might also be measuring any combination of the above, like maybe case 2 turns into case 1, and potentially we even run into case 3 at the same time when we get a really bad interleaving and a really big outlier. But I think any combination is fair game when comparing schedulers, because I'm measuring the same thing on all schedulers.

If we think about it like this, it's really really bad that this measurement can take multiple milliseconds. It's not something I have ever seen before. My previous experience was on Windows, Xbox One (which uses a modified version of Windows) and Playstation 4 (which uses a modified version of FreeBSD) and none of them would ever show anything like this. If all threads but one are yielding, that one thread is definitely going to run. And no way would it take a millisecond for that to happen. (of course it usually doesn't take that long on Linux either, but if the outliers are too big, we really care about them)

Thanks for responding, I really have a lot of admiration for you and your work, (and I would have wished that our first conversation wasn't this negative... but I can't change what I found) - Malte"

This is why development is slow and hard on tough problems. Experienced people tend to be standoffish and hard to work with while younger developers that need the mentoring suffer in solitude. I try to work differently, but when I'm already otherwise engaged (and having suffered myself through decades of such problems), it's tough to change gears, slow down, and do the right thing - it's a common struggle for all of us.

This is also why you have to have understanding to allow yourself to give such new technology cutting edge projects the time and resources to progress. It takes time. Nothing comes out perfectly formed the first time through. You can't wait for perfection to release and gain valuable feedback for further refinements.

Here is Malte's blog post:

Measuring Mutexes, Spinlocks and how Bad the Linux Scheduler Really is
by Malte Skarupke
https://probablydance.com/2019/12/3...ks-and-how-bad-the-linux-scheduler-really-is/

 

It's not that he's hard to work with (but yeah, he scares away snowflakes), it's just that the game in question isnt coded well. Not a huge issue onto itself, but the dev then had to go on and blame someone else for their problems. Hence the virtual whipping from Torvalds, who actually knows a thing or two about how Linux works.

If you want to blame something else for stadia's problems like this dev, have at it. But I'd rather trust the word of someone far smarter than this dev. At least the dev responded well to the criticism, somewhat. Going on to blame Torvalds for being "hard to work with" is a bs excuse since Torvalds doesnt actively work for a game developer nor regualrly helps them out; his speciality was the kernel, that's it. This is ignoring that various games on Steam and elsewhere dont have any issues with the Linux scheduler and also implies that it's ok to blame unrelated people for you problems. I might as well tell off Steve Woz for my hypothetical macOS game's problems.

 

I'm not blaming the dev for Stadia problems, I posted it to give a glimpse into the problems found when optimizing games - and in this case for Stadia.

You're are far more cynical than I am, I try to find solutions for a long time and rarely have time to worry about blame. I make things work in situations where many have tried and failed.

That's the insight I try to bring to problems that are easy to obfuscate and create an emotional response - like you giving up prematurely on Stadia - where there is plenty of room for solutions left to implement to smooth out the implementation and user experience.

Name calling, wasting time affixing blame, stubbornly trying to justify a negative position by constantly spinning issues into far worse than they need to be, are signs of poor problem management.

That's Linus's problem, which he freely admits he has, and doesn't care enough about others to control his bad attitude. Linus would rather skewer someone than explain the error of their actions or situation and give them direction on how to correct it.

Those approaches don't solve problems in groups - it creates a situation where only one developer - Linus in this case - will stick around and work on fixing things. The others are scared off or quite rightly quit dealing with such abuse and find other things to do.

It doesn't matter that Linus makes a technical point that is valid, it matters that he craps on the person instead of interacting constructively to solve the problem overall. Which I don't think he did. Even if the well known problem of implementing user level locking is imperfect vs kernel locking, Linus didn't investigate the application and it's needs to suggest a specific directed solution.

Linus ignored the person and the specific need and shot back focusing on demeaning the person and pointing out an obvious fallacy, being completely non-constructive. That's Linus's method for controlling the Linux Kernel development, and has been for many years.

When Linus was starting out, from the first post on UUNET in the Minux group, he was a reasonable person, but later on he lost his edge and devolved into madness - at least from my point of view.

Stadia needs everyone to pull their weight to make it work, there is no benefit in spending time wasting effort on dealing with Linus, I hope Malte Ignores Linus and works with the current leads on those Linux areas he needs changes in to optimize his game development(s).

 

I've not "given up" on Stadia per se, just that I don't see Google actually fixing it correctly nor the concept as a whole going anywhere outside of games that are very tolerant of input lag and delayed response times from the player (like Civilization and other turn-based games).

I'm pretty sure Linus *did* explain the issues in that game's code in his statement:

It's in typical Linus fashion, but it does explain the issue: Rolling your own locks doesn't work, especially if you don't take the time to understand how locks work and how to implement them correctly for the situation given. Using another "you shouldn't do this on your own" analog, this would be similar to creating your own encryption scheme and expecting it to just work and be secure (it won't be).

I'm not entirely sure you're familiar with how Linux kernel development works, otherwise you'd know that it's like any other FOSS project that has a core of "hardcore" devs that stick with it for a long time, many more on-and-off devs that do this and that, and Linus hasn't actively worked on the kernel for a while (busy doing other things and he mostly does code review for the kernel these days).

Anyway, we have one game dev shifting blame to someone else, several other games having issues on the platform with regard to previously-discussed Stadia problems, and Google not doing much to fix those for the time being. Meanwhile, non-Stadia Linux games seem to be doing just fine, so blaming the kernel isn't productive. Which leads me to believe that, if other Linux games seem to be doing fine and Stadia games seem to be having issues, it *might not* be Linux that's the problem here but instead the game developers and/or Stadia that are the problem. Figuring out what/who is the problem isn't an issue, but a first step in figuring out how to fix something; if we're going to blame something that isn't the problem, how is anyone expected to actually fix the issue? If anything, the dev should be looking towards Google for assistance; it's their platform after all.

---

To preempt any "you hate Google" or "you hate cloud" comments: no, I don't. Google makes some pretty fantastic stuff (search engine, maps, gmail) and not-so-fantastic stuff (google+, wave, Youtube). Likewise, there are various cloud services that I do use (again Google, but also Amazon, Steam and friends, Discord, etc). I'm critical of things that are not good and/or ran poorly, doesn't matter what it is or who it is.

 

You have not tried geforce now obviously.

input lag for stadia is around 70-120 ms and that is unacceptable..but once they hit 30ms or less and they will fps will be able to be played competitively.

with geforce now Im getting 35ms to the north east servers and there appears to be no lag well nothing a human can detect and this is why I said this is the future of gaming and google yes the multi billion dollar company knows this also. people need to try the steaming out versus arguing against something they have no insight on. I highly recommend you try geforce now and then get back to me and tell the world this is not indeed the future.

I played project cars 2 with a keyboard and streamed it albeit not perfectly however I came in first and as of today not many people can come in first using a keyboard in the game ala the million project cars 2 videos.

This ignorant or not genius or not average or not is the future

 

The one game that offered, "theoretically the best use case for [Stadia]" has seen its player base evaporate.

https://www.forbes.com/sites/paulta...-by-more-than-half-since-launch/#470dd6ba2604

 

you do realize the internet itself had a rocky start

https://en.wikipedia.org/wiki/History_of_the_Internet

 

Thanks for that. I've been trying to say that all along. I've been on the Internet before it was the Internet, changing protocols, changing services, changing applications - NCP -> IP/TCP, hosts.txt => DNS, 9600 baud to 1Mbit WAN connectivity, wired to satellite to wireless as big as a refrigerator down to the size of an M.2 plug-in card + a few wires.

From 1978 to 2020, that's a long patient ride to where we are today.

Why worry about Staida's glitches after a month? Get an account and start using it, wait a few years, look back and smile.

That's what I do with all of this stuff, smile.

 

Sure, and when Stadia stops being crap, you'll hear a more positive tone out of the people on the internet not too happy with it right now. Until then, it's crap and Google needs to fix it.

 

If you shell out $130 + monthly for my account, I'll be glad to give it a go. Personally, I don't have the disposable income at the moment to mess around with a beta product.

 

Even when the Internet was "crap" it was fun. If you don't think new technology is fun, what are you doing here?

Fair enough, so you can afford to be patient and wait for it to become affordable.

Why complain about something you can't afford even if it was a perfect implementation?

You should be happy it's not perfect at $130/month, now Google is using both things to reduce the load on the technology in it's infancy, and later as Google builds out more infrastructure the price can drop and allow more people in to the Stadia system.

It's supposed to be "free" soon, + game costs, then game prices can drop to be competitive, then maybe a "game service" that bundles access to games can be supported by the built out infrastructure, so maybe then it will be worth "playing with"?

 

I like *practical, working* technology, not half-baked products or passion products (like Bitcoin, for example). New stuff excites me only after it starts working correctly and meets an actual need.

Well, ignoring the income bashing you just did, if I'm going to spend that sort of money on a product, I would like for it to work. I've spent money on beta products before (Kerbal Space Program, for instance), but those were far, far cheaper than Stadia and fell within my own personal "f*ck it I don't care" throw-away budget. I do start to care once you're asking me to pay more than that. In a year when it becomes cheaper and *hopefully* it's not garbage, maybe I can give it a try then. Personally though, I'm not expecting much from it as compared to what I can do today with my gaming desktop and laptop (which are truly free now that I've long since paid off those credit card charges).

--------

To comment on the Internet example, the early days of the internet were ****, people rightfully complained about the issues, and later on they were fixed as a result of these complaints and better tech coming along.

For another example, all-electric cars have been complete garbage until very recently. What's the point of spending $20k-$40k on a Nissan Leaf when it could only carry you around 30 miles or so? Something like a Leaf was rightfully bashed in with criticism and it failed pretty badly as a result, as all bad products should. Only later after a lot of this criticism has been levied out and tech got to the point where it was actually feesable did we start seeing electric cars that are actually worth a d*mn.

So far we have something similar going on with OnLive, Now, and Stadia. Maybe it could work in the future, but now...? Given nVidia's and Google's tendencies to ignore criticism for their products, I have my doubts that even the future will work out well for this stuff.

 

And yet, lots of people bought all of those things in the early days where it was an extra burden to use them - lose money or suffer other inconveniences with a lessor or different than common expectations made acceptable.

I admire all of those that bought the first electric cars... in the late 1800's and early 1900's along with the electric trams and other electric transport that was burgeoning at the time - and would have continued to rapidly progress if not for "big money" which brought "big oil" that took us to where we are today.

Truly we should have been complaining about the smoke belching vehicles subsidized by cheap oil subsidized by our government after "big money" figured out how to pass on the burden to the government.

There's always a lot more to a situation than appears from the outside.

Often all we can do is take what's given to us as an option and if we want what it promises to provide when it's optimized someone has to fund it until it is more desirable to the population at large.

Fund it till you make it, then grow like you own it, reaping the benefits yourself and for everyone else later down the road. Knowing you had a small part in it's eventual success and continued operation, or knowing you did something - you participated when others sat on the sidelines and tried to shoot it down before it even got started.

Vote with your wallet means - hey, I want this to work better and better, but for now I'll support it with my investment in time and money until everyone else see's it's value.

If you want to wait until everyone has helped get it off the ground, you don't value investing in the future you want to live in, then wait for others to help make things you want succeed and then reap the benefits.

Maybe even next month, or the following year you'll want to participate in the success when it's where you want it to be, why participate in trying to kill it before it has a chance to do that?

 

Stadia deserves to die because it's a crap product with a crap business model. GeForce Now and PlayStation Now are superior products with far more consumer-friendly business models.

 

Or support the competition. saturnotaku mentioned GeForce Now and PlasyStation Now, for example. I'm not entirely familiar with those product, but do know that GeForce Now has been around for longer and supposedly is better polished.

Save the melodrama for something that's actually important. This is a videogame platform, not a medical breakthrough or stocks in a startup.

At any rate, there's nothing wrong with waiting for something to get good before spending money on it, right? You can't seriously be advocating for people to spend their money on a half-baked or failing product because you like it.

If you want to try to "convert me" to Stadia, the least you could do is foot the bill for me and give me a free Stadia account and library