-
Notifications
You must be signed in to change notification settings - Fork 890
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
macos-12: timestamps differ by 182 seconds - check your system clock #2996
Comments
We are experiencing the same issue on macOS-13 runners. Simple re-running fixes the problem, but it occurs occasionally. |
Also seeing this randomly:
And, when using either
Error verbatim: |
Seeing the same issue, interestingly around the same time skew: 183 and 187 in different instances. |
Support got back to me privately, this is what they recommend as a workaround while engineering investigates: steps:
- name: Sync clock
run: sudo sntp -sS time.windows.com Surprisingly, the commend succeeds. I asked why, and they said that Since the issue is sporadic, I haven't had enough builds yet to know if it fixes the problem.
|
Thanks @b099l3 for this workaround. I did some test runs on macos-13, and am getting highly variable results of 0 - 204 seconds time difference. Interestingly, the differences are both positive and negative: Run sudo sntp -sS time.windows.com |
Hidden, outdatedQuoting https://github.com/fastlane/fastlane/pull/21583#issuecomment-1833616129:
... suggesting that the provided workaround from GitHub support only slightly improves this problem, but is still unacceptable. At time of writing this, I still do not have enough runs since applying this patch to make a similar report. What's particularly interesting about fastlane's bug report is it does not have the same timestamp message in the error output, suggesting that this bug may be affecting other API calls with slightly different error messages. For example, I have also observed the following error occur in the logs: Run xcrun notarytool submit --wait "my-file.pkg" \
Conducting pre-submission checks for my-file.pkg and initiating connection to the Apple notary service...
- Error: HTTP status code: 401. Unable to authenticate. The request cannot be processed at this time. Please try again later.
- Ensure that all authentication arguments are correct. ... which may be related to the fastlane error |
Hi, @tresf , sorry, but you are misquoting my testing. I only have succesful results with this workaround:
The time differences before syncing is reported above, and in summary, I observed time differences from -204 to +100 seconds, and occasionally <1 second. All runs were successful, thanks to the 'Sync clock' step. The success/failure rates were referencing the open PR to Fastlane: Hope that clarifies things :-) |
It does, thank you. I'll edit the post to hide the irrelevant parts. |
Hey folks, thanks for the report. Our hosted mac team is working on a fix, they will follow up in this thread. |
Any updates on this? It's really getting annoying. Our internal deployment fails every few days due to this issue. |
I can't speak on behalf of GitHub support (that's who I reached out to first, because I pay GitHub for these services), but this seems to be the accepted workaround: #2996 (comment). The original question has been amended to include this answer. |
I can confirm that I haven't seen this problem since using the |
Hey folks, I'm from the hosted macOS runner team, we have rolled out fixes for the root causes of the time drift, and don't expect to see this time drift anymore. There's a slight caveat that some runners that were registered before the fixes were rolled out may still see some impact, but that caveat should expire in a few hours. That being said, please |
Should this be marked as closed or would you rather monitor it? |
@tresf we'll monitor for a bit longer and will then close the comment. Thanks again for bringing this to our attention and for bearing with us. |
This reverts commit a5d01ae.
UPDATE: Potential workaround:
Original post:
There is a private support ticket (2444231) open for this issue, but I wanted to file a public report incase this problem is affecting others.
Quoting:
Describe the bug
Intermittent signing failures calling
|xargs codesign --force -s P5DXXXXXXX --timestamp --options runtime
on 33 files.- 2023-11-20T15:02:25.6218970Z [exec] ./Contents/Frameworks/myfile.dylib: timestamps differ by 182 seconds - check your system clock
To Reproduce
Unsure how to reproduce.
sudo sntp -sS time.apple.com
to force synchronization, but the command fails onmac-12
Quoting my colleague:
Expected behavior
Runner Version and Platform
OS of the machine running the runner? OSX/Windows/Linux/...
What's not working?
codesign
command, intermittentlyWorkaround
Job Log Output
Runner and Worker's Diagnostic Logs
If applicable, add relevant diagnostic log information. Logs are located in the runner's
_diag
folder. The runner logs are prefixed withRunner_
and the worker logs are prefixed withWorker_
. Each job run correlates to a worker log. All sensitive information should already be masked out, but please double-check before pasting here._diag
folder. I'm happy to provide more information.The text was updated successfully, but these errors were encountered: