Added constants for 12.0 support · acidanthera/CpuTscSync@e43a70e

mishurov · 2021-07-09T04:24:20Z

Doesn't work in 12. There're literally two lines of code that matter. And they don't work for some reason.

uint64_t tsc = rdtsc64();
mp_rendezvous_no_intrs([](uint64_t *tscp) { wrmsr64(MSR_P5_TSC, tscp); }, &tsc);

https://github.com/apple/darwin-xnu/blob/8f02f2a044b9bb1ad951987ef5bab20ec9486310/osfmk/i386/proc_reg.h#L467
https://github.com/apple/darwin-xnu/blob/8f02f2a044b9bb1ad951987ef5bab20ec9486310/osfmk/i386/mp.c#L464

TSCAdjustReset works but it does this, and I'm not sure if it is a correct way to go

#define MSR_IA32_TSC_ADJUST 0x0000003b
mp_rendezvous_no_intrs([](){ wrmsr64(MSR_IA32_TSC_ADJUST, 0); }, NULL);

https://github.com/torvalds/linux/blob/64a925c9271ec50714b9cea6a9980421ca65f835/arch/x86/kernel/tsc_sync.c

I wonder, maybe it should be a part of OpenCore itself, not as a kext.

lvs1974 · 2021-07-09T05:31:05Z

@mishurov: there is a part of OpenCore responsible for sync - TscSyncTimeout, you can try it.

Attempts to perform TSC synchronisation with a specified timeout.
The primary purpose of this quirk is to enable early bootstrap TSC synchronisation on some server and laptop models when running a debug XNU kernel.
For the debug kernel the TSC needs to be kept in sync across the cores before any kext could kick in rendering all other solutions problematic.
The timeout is specified in microseconds and depends on the amount of cores present on the platform, the recommended starting value is 500000.
This is an experimental quirk, which should only be used for the aforementioned problem.
In all other cases, the quirk may render the operating system unstable and is not recommended. The recommended solution in the
other cases is to install a kernel extension such as VoodooTSCSync, TSCAdjustReset, or CpuTscSync (a more specialised variant of VoodooTSCSync for newer laptops).
Note: This quirk cannot replace the kernel extension because it cannot operate in ACPI S3 (sleep wake) mode and because the UEFI firmware only provides very limited multicore support which prevents precise updates of the MSR registers.

You can also check quirks InitialTSC and ProvideCurrentCpuInfo.

vit9696 · 2021-07-09T06:15:23Z

if I remember correctly, the reason this stopped working is CPU core matching. But I do not have macOS 12 to check.

lvs1974 · 2021-07-09T06:35:15Z

@vit9696: I don't have macOS 12 either.
@mishurov: could you upload AppleACPIPlatform.kext from macOS 12, and may be you could save IOReg to check matching.
CpuTscSync/VoodooTscSync uses IOProviderClass = AppleACPICPU for matching.

mishurov · 2021-07-09T07:16:28Z

It is not just CPU matching, if I'm not wrong. TSCAdjustReset matches with IOCPUNumber 0. But when I try to execute rdtsc and rendezvous its value on the cores, it panics. It works if I do wrmsr64(MSR_IA32_TSC_ADJUST, 0) first though.

I'll share IOReg tomorrow on the weekend, I have to work now unfortunately.

lvs1974 · 2021-07-09T07:21:19Z

@mishurov: if rdtsc causes panic, would be great to see a panic report (OpenCore can save such reports on ESP).

mishurov · 2021-07-09T08:12:21Z

@lvs1974

The kext, ioreg and the panic log when use TSCAdjustReset with the code from VoodooTSCSync.cpp.

AppleACPIPlatform.kext.zip
ioreg.txt
panic-2021-07-09-073926.txt

lvs1974 · 2021-07-09T08:40:08Z

@mishurov: as far as I can see - this panic has nothing common with calling rdtsc, it was just an assert in kernel, in method thread_dispatch.

		assertf(processor->last_dispatch >= self->last_made_runnable_time,
		    "Non-monotonic time? dispatch at 0x%llx, runnable at 0x%llx",
		    processor->last_dispatch, self->last_made_runnable_time);

I think this panic is a consequence of non-working TSC reset in hu.interferenc.TSCAdjustReset (most likely it was loaded by the moment this panic happened).
And in some cases reset does not help, sync is needed.

Try to turn on quirk TscSyncTimeout in OpenCore, and get boot-log with debug version of CpuTscSync and boot-arg -cputsdbg.

And try to check whether the CpuTscSync kext is loaded.

lvs1974 · 2021-07-09T08:52:19Z

@mishurov: I think you use TSCAdjustReset in a wrong way.
This kext matches to CPU0 according to your ioreg. But it must always match to the last CPU (CPU7 in your case).
You have to fix it in Info.plist (see IOCPUNumber).

VoodooTscSync/CpuTscSync does not need this adjustment.

mishurov · 2021-07-09T10:03:29Z

@lvs1974
TSCAdjustReset doesn't work if I set 7 or 3

I compiled the debug version from the latest commit, enabled TscSyncTimeout and now it boots without hangs. I can't reproduce the bug anymore. VoodooTSC matches to CPU3, it seems reasonable because my system has 2 physical cores and 4 logical cores.

lvs1974 · 2021-07-09T10:16:52Z

@mishurov: have you checked that CpuTscSync is loaded?
Try to sleep/wake and get logs from CpuTscSync.

mishurov · 2021-07-09T10:26:18Z

@lvs1974 sleep/wake didn't work but I can't say if it is because of CpuTscSync or not. I disabled sleep long ago and I'm not sure if it worked previously. I use Linux primarily, macOS is only to debug iOS apps from time to time. Sleep brought only problems if I, for example, leave the laptop to update XCode and it goes to sleep and cuts the connection.

lvs1974 · 2021-07-09T10:29:44Z

@mishurov: if you don't use sleep, quirk TscSyncTimeout might be enough for you.

mishurov · 2021-07-09T10:37:28Z

@lvs1974 that is what I thought initially, it hangs without the kext.

Apple rewrote the Bluetooth stack in Monterey moving it into the userspace, now Intel wireless cards work without bugs, on the previous versions they couldn't connect to BT4 LE HID devices. Also BT5.0. That was the only reason to update otherwise I was pretty happy with Catalina.

lvs1974 · 2021-07-09T10:43:33Z

@mishurov: hangs without CpuTscSync or TSCAdjustReset?

mishurov · 2021-07-09T10:46:36Z

@lvs1974 it hangs without a kext. Either TSCAdjustReset or CpuTscSync should be loaded.

vit9696 · 2021-07-09T11:40:55Z

So, am I right that the issue is late matching? I.e. CpuTscSync matches (and starts) very late, and thus it is too late to fix up the TSC MSR?

lvs1974 · 2021-07-09T11:44:34Z

@vit9696: I am not sure. In panic log we can see that TSCAdjustReset was already loaded...

vit9696 · 2021-07-09T11:46:44Z

Is it initialised though?

lvs1974 · 2021-07-09T12:09:18Z

@vit9696: we don't know.
@mishurov: can you try attached kext, please?
CpuTscSync-1.0.4-DEBUG.zip

mishurov · 2021-07-09T12:37:28Z

@lvs1974 it works, the same way as my build from the master branch. It needs TscSyncTimeout (500000) enabled, otherwise it hangs. TscSyncTimeout alone without a kext also hangs. TSCAdjustReset doesn't need TscSyncTimeout enabled.

lvs1974 · 2021-07-09T12:41:17Z

TSCAdjustReset doesn't need TscSyncTimeout enabled.

Strange. You attached panic report with loaded TSCAdjustReset.
And now you say it works.

mishurov · 2021-07-09T12:45:38Z

@lvs1974 the original TSCAdjustReset always works with IOCPUNumber=0. I just resets MSR_IA32_TSC_ADJUST to 0 on all CPUs. It panicked when I tried to use with the code from Voodoo which reads TSC rdtsc64() and sets the value on other CPUs.

lvs1974 · 2021-07-09T12:48:12Z

@mishurov: I see now, thank you.
Could you try one more version without TscSyncTimeout, please?
CpuTscSync-1.0.4-DEBUG.zip

mishurov · 2021-07-09T13:10:03Z

@lvs1974 it hangs. I can't enable OC logging into a file. I set AppleDebug to true and add log=1 boot arg, but there's nothing in the ESP, only panic logs.

lvs1974 · 2021-07-09T13:28:45Z

@mishurov: let's try modified VoodooTSCSync.
VoodooTSCSync.kext.zip

mishurov · 2021-07-09T13:32:56Z

I've figured how to enable logging, bitmask for target. Here's the log with the previous kext and -cputsdbg
opencore-2021-07-09-132956.txt

P.S. Voodoo hangs too
opencore-2021-07-09-133547.txt

lvs1974 · 2021-07-09T13:46:36Z

@mishurov: and before macOS 12 CpuTscSync or VoodooTscSync worked and you did not have any issues?
Attached kext will produce a panic in VoodooTSCSync::probe, so in panic report you will see string "VoodooTSCSync::probe.....................................".
I just want to be sure this kext is loaded.
VoodooTSCSync.kext.zip

And opencore logs don't help a lot, they contain only info before starting macOS kernel...

mishurov · 2021-07-09T13:57:35Z

@lvs1974 yes, I used CpuTscSync and it worked fine. The problem appeared when I installed update from Catalina to Monterey (I didn't use Big Sur) and booted into installer/updater.

Yes, the kext produces the panic.

lvs1974 · 2021-07-09T14:29:14Z

@mishurov: the last attempt - match VoodooTSCSync only to CPU 0 (no panic):
VoodooTSCSync.kext.zip

mishurov · 2021-07-09T14:33:21Z

@lvs1974 it hangs unfortunately.

lvs1974 · 2021-07-09T16:43:04Z

@mishurov: I tried to combine functionality of VoodooTSCSync and TSCAdjustReset in attached version of CpuTscSync (it always matches to CPU=0):
CpuTscSync-1.0.4-DEBUG.zip

mishurov · 2021-07-09T16:51:35Z

@lvs1974 no luck.

I was able to append to TSCAdjustReset

uint64_t tsc = rdtsc64();
mp_rendezvous_no_intrs(stamp_tsc, &tsc);

after

mp_rendezvous_no_intrs(reset_tsc_adjust, NULL);

But I don't know if it makes any sense. As I understand, practically the only issue with TSCAdjustReset is sleep/wake and I can't test it.

lvs1974 · 2021-07-09T16:55:41Z

@mishurov: attached CpuTscSync does this:

// Update MSR on all processors.
void VoodooTSCSync::doTSC()
{
	uint64_t tsc = rdtsc64();
	SYSLOG("cputs", "current tsc from rdtsc64() is %lld. Rendezvouing..\n", tsc);
	
	// call the kernel function that will call this "action" on all cores/processors
	mp_rendezvous_no_intrs(reset_tsc_adjust, NULL);
	mp_rendezvous_no_intrs(stamp_tsc, &tsc);
	tsc_synced = true;
}

lvs1974 · 2021-07-09T16:58:55Z

This version resets TSC and after syncs (reads TSC after reset):
CpuTscSync-1.0.4-DEBUG.zip

void VoodooTSCSync::doTSC()
{
	SYSLOG("cputs", "reset tsc. Rendezvouing..\n");
	mp_rendezvous_no_intrs(reset_tsc_adjust, NULL);

	// call the kernel function that will call this "action" on all cores/processors
	uint64_t tsc = rdtsc64();
	SYSLOG("cputs", "current tsc from rdtsc64() is %lld. Rendezvouing..\n", tsc);
	mp_rendezvous_no_intrs(stamp_tsc, &tsc);
	tsc_synced = true;
}

mishurov · 2021-07-09T17:17:01Z

I just tried to read tsc before calling reset_tsc_adjust in TSCAdjustReset, it hangs too. The latest kext hangs as well for some reason, may be because it is a Lilu plugin, I don't know.

Although it went past "Processor Id = 1 ... Enabled"

lvs1974 · 2021-07-09T17:22:26Z

@mishurov: this kext does only reset, no sync:
CpuTscSync-1.0.4-DEBUG.zip
Just want to be sure it can work as well as TSCAdjustReset.

mishurov · 2021-07-09T17:30:40Z

@lvs1974 yep, this one works as well as TSCAdjustReset.

lvs1974 · 2021-07-09T17:34:30Z

@mishurov: so, the problem is only with writing values to MSR_IA32_TSC.

lvs1974 · 2021-07-09T17:37:52Z

We can try one more thing:

//stamp the tsc
extern "C" void stamp_tsc(void *tscp)
{
	wrmsr64(MSR_IA32_TSC_ADJUST, *reinterpret_cast<uint64_t*>(tscp));
	wrmsr64(MSR_IA32_TSC, *reinterpret_cast<uint64_t*>(tscp));
}

CpuTscSync-1.0.4-DEBUG.zip

mishurov · 2021-07-09T17:57:55Z

@lvs1974 it doesn't work, hangs again.

In Linux they read from MSR_IA32_TSC_ADJUST, not from rdtsc64()

s64 curval;
rdmsrl(MSR_IA32_TSC_ADJUST, curval);

But they write another value

Also this comment is interesting:

On the boot cpu we just force set the ADJUST value to 0 if it's
non zero. We don't do that on non boot cpus because physical
hotplug should have set the ADJUST register to a value > 0 so
the TSC is in sync with the already running cpus.

Also don't force the ADJUST value to zero if that is a valid value
for socket 0 as determined by the system arch. This is required
when multiple sockets are reset asynchronously with each other
and socket 0 may not have an TSC ADJUST value of 0.

https://github.com/torvalds/linux/blob/64a925c9271ec50714b9cea6a9980421ca65f835/arch/x86/kernel/tsc_sync.c#L84

P. S. Probably it explains the wake problem, maybe it should reset ADJUST only on boot, but during wake it should execute the Voodoo code or something more sophisticated.

lvs1974 · 2021-07-09T18:39:36Z

@mishurov: attached kext resets TSC only once, at start.
And it syncs TSC only if macOS woke up.
CpuTscSync-1.0.4-DEBUG.zip
Sources:
Archive.zip

mishurov · 2021-07-09T19:00:16Z

@lvs1974 it boots okay. Sleep doesn't work on my machine but I don't know if it worked in the first place, could be caused by other hardware and settings, so I can't give 100% assurance on that matter.

lvs1974 · 2021-07-09T19:04:54Z

@mishurov: and this version works on my hack - boot + sleep/wake.

lvs1974 · 2021-07-09T20:23:09Z

@mishurov: could you try one more version: matching to the last CPU.
CpuTscSync-1.0.4-DEBUG.zip

mishurov · 2021-07-10T01:03:11Z

@lvs1974 it doesn't work again, the on-screen log again stops during the CPUs detection.

Shinji3rd · 2021-07-16T07:06:24Z

Hi! If you need another tester, let me know, maybe I can help. I have an i7 7500U, and I was using CpuTscSync in Big Sur. It all worked perfectly, including sleep/wake (I do use the sleep function). Now I updated to Monterey, CpuTscSync doesn't work (CPU kernel panic), so I'm not injecting it and I replaced it with TSCAdjustReset. This last one works, but as others mentioned before, waking from sleep is broken (it goes to sleep, but it cannot wake, it shows a black screen and the fans starts spinning at max).

mishurov · 2021-07-16T19:31:13Z

@Shinji3rd I can't test sleep.

Try the build from this comment, it executes another code on wake
e43a70e#commitcomment-53283096

Shinji3rd · 2021-07-16T19:39:35Z

I went ahead and tested it. It boots, but sleep is still broken with that one (exactly the same that happens with TSCAdjustReset). Another thing that may be worth mentioning is that the boot process is too long (~2 minutes on Monterey, and it was ~20 secs on Big Sur), but may be caused by another thing.

vit9696 · 2021-07-16T22:38:39Z

Please try https://github.com/acidanthera/CpuTscSync/tree/macos-star, also reply in acidanthera/bugtracker#1676.

-Original file line number
+Diff line change
@@ -1,5 +1,7 @@
     CpuTscSync Changelog
     ===================
+    #### v1.0.4
+    - Added constants for 12.0 support
     #### v1.0.3
     - Added MacKernelSDK with Xcode 12 compatibility
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -35,7 +35,7 @@ PluginConfiguration ADDPR(config) { @@
     	bootargBeta,
     	arrsize(bootargBeta),
     	KernelVersion::MountainLion,
-    	KernelVersion::BigSur,
+    	KernelVersion::Monterey,
     	[]() {
     		cpuf.init();
     	}
@@ Expand Down @@

Commit

There are no files selected for viewing

50 comments on commit e43a70e

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

vit9696 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

vit9696 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

vit9696 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021 • edited Loading

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

mishurov commented on e43a70e Jul 9, 2021

Choose a reason for hiding this comment

lvs1974 commented on e43a70e Jul 9, 2021

50 comments on commit `e43a70e`

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021 •

edited

Loading

vit9696 commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021 •

edited

Loading

lvs1974 commented on `e43a70e` Jul 9, 2021 •

edited

Loading

lvs1974 commented on `e43a70e` Jul 9, 2021 •

edited

Loading

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021 •

edited

Loading

mishurov commented on `e43a70e` Jul 9, 2021 •

edited

Loading

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

vit9696 commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

vit9696 commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021 •

edited

Loading

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021 •

edited

Loading

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021 •

edited

Loading

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 9, 2021 •

edited

Loading

lvs1974 commented on `e43a70e` Jul 9, 2021 •

edited

Loading

mishurov commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

lvs1974 commented on `e43a70e` Jul 9, 2021

mishurov commented on `e43a70e` Jul 10, 2021

Shinji3rd commented on `e43a70e` Jul 16, 2021 •

edited

Loading

mishurov commented on `e43a70e` Jul 16, 2021 •

edited

Loading

Shinji3rd commented on `e43a70e` Jul 16, 2021 •

edited

Loading

vit9696 commented on `e43a70e` Jul 16, 2021