Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul of SlaveLaunchLogs #517

Merged
merged 25 commits into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8095453
agent-logs sketch
jglick Feb 29, 2024
ac39d18
`SlaveLaunchLogsTest.onlineInboundAgent`
jglick Feb 29, 2024
a70c20d
`SlaveLaunchLogsTest.offlineAgent`
jglick Feb 29, 2024
98c04e5
Javadoc
jglick Feb 29, 2024
0cd0f81
Merge branch 'master' of https://github.com/jenkinsci/support-core-pl…
jglick Mar 1, 2024
3d0a97a
Exploring `SlaveLaunchLogs` behavior
jglick Mar 1, 2024
cdf931f
More `SlaveLaunchLogsTest`
jglick Mar 1, 2024
0b192d3
`SlaveLaunchLogsTest.passwords`
jglick Mar 1, 2024
e28c8c3
Worked out a better `SlaveLaunchLogs`, but depends on patch to `Slave…
jglick Mar 1, 2024
83ed22f
Need to flush logs also for `deletedAgent`
jglick Mar 1, 2024
4a2adc4
Reverting changes extracted to #518
jglick Mar 1, 2024
154adca
Setting a timestamp, switching category
jglick Mar 1, 2024
92cda57
RC
jglick Mar 4, 2024
d698388
No need to assert that `SupportTestUtils.invokeComponentToString` is …
jglick Mar 4, 2024
93f8c92
`Security2186Test` failure caused by renamed bundle entry
jglick Mar 4, 2024
98ee7ab
File handle leak caught by Windows tests
jglick Mar 4, 2024
6fd40f8
Merge branch 'master' of https://github.com/jenkinsci/support-core-pl…
jglick Mar 4, 2024
72beede
Better handling of rotated logs
jglick Mar 4, 2024
d072846
More robust way to wait for `Connection terminated` message
jglick Mar 4, 2024
9169e04
`SlaveLaunchLogsTest.offlineAgent` still flaky on Windows
jglick Mar 4, 2024
2fa5cb1
https://github.com/jenkinsci/jenkins/pull/9009 released
jglick Mar 5, 2024
a573c50
Merge branch 'master' of https://github.com/jenkinsci/support-core-pl…
jglick Mar 18, 2024
fc62f58
Working around lack of JENKINS-72799 to avoid requiring a weekly core
jglick Mar 18, 2024
c9d6e95
Merge branch 'master' into agent-logs
Dohbedoh Mar 18, 2024
5a924b4
SpotBugs
jglick Mar 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package com.cloudbees.jenkins.support.api;

import com.cloudbees.jenkins.support.filter.PasswordRedactor;
import com.cloudbees.jenkins.support.impl.SlaveLaunchLogs;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileInputStream;
Expand All @@ -11,6 +12,9 @@
import java.util.stream.Collectors;
import org.apache.commons.io.IOUtils;

/**
* @see SlaveLaunchLogs
*/
public class LaunchLogsFileContent extends FileContent {

public LaunchLogsFileContent(String name, String[] filterableParameters, File file, long maxSize) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@

/**
* Matches agent log files in an interval of time
* @see SlaveLogs
*/
class LogFilenameAgentFilter implements FilenameFilter, Serializable {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@
return;
}

// TODO this is not necessarily the right root for workDir.

Check warning on line 48 in src/main/java/com/cloudbees/jenkins/support/impl/NodeRemoteDirectoryComponent.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: this is not necessarily the right root for workDir.
// Engine.workDir says that the workDir “should” be the agent root but this is not enforced.
// Safer to use Engine.current().internalDir.
FilePath rootPath = node.getRootPath();
if (rootPath == null) {
LOGGER.log(Level.WARNING, "Node " + node.getDisplayName() + " seems to be offline");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,12 @@
tasks.add(new java.util.concurrent.Callable<List<FileContent>>() {
public List<FileContent> call() throws Exception {
List<FileContent> result = new ArrayList<FileContent>();
// TODO presumes that WinSW’s %BASE% would be the remoteFS as in

Check warning on line 213 in src/main/java/com/cloudbees/jenkins/support/impl/SlaveLogs.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: presumes that WinSW’s %BASE% would be the remoteFS as in
// https://docs.cloudbees.com/docs/cloudbees-ci-kb/latest/client-and-managed-controllers/how-to-install-windows-agents-as-a-service
// and that
// https://github.com/winsw/winsw/blob/6cf303c1d3fbe1069d95af230b8efa117d29cdf2/src/WinSW.Core/Configuration/XmlServiceConfig.cs#L273
// is not overridden from e.g.
// https://github.com/winsw/winsw/blob/e4cf507bae5981363a9cdc0f7301c1aa892af401/samples/shared-directory-mapper.xml#L8
final Map<String, File> logFiles = logFetcher.forNode(node).getLogFiles(rootPath);
for (Map.Entry<String, File> entry : logFiles.entrySet()) {
result.add(new FileContent(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// TODO delete?

Check warning on line 1 in src/main/java/com/cloudbees/jenkins/support/impl/SmartLogCleaner.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: delete?
package com.cloudbees.jenkins.support.impl;

import com.cloudbees.jenkins.support.SupportPlugin;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// TODO delete? (9aeca363ce4812f5f6c084d673e8d8bcdb266498)

Check warning on line 1 in src/main/java/com/cloudbees/jenkins/support/impl/SmartLogFetcher.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: delete? (9aeca363ce4812f5f6c084d673e8d8bcdb266498)
package com.cloudbees.jenkins.support.impl;

import com.cloudbees.jenkins.support.SupportPlugin;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
/**
* Matches log files from winsw.
*
* @see <a href="https://github.com/kohsuke/winsw/blob/master/LogAppenders.cs">LogAppenders.cs</a>
* @see <a href="https://github.com/winsw/winsw/blob/e4cf507bae5981363a9cdc0f7301c1aa892af401/src/WinSW.Core/LogAppenders.cs#L169-L170">LogAppenders.cs</a>
* @see SlaveLogs
* @author Kohsuke Kawaguchi
*/
class WinswLogfileFilter implements FilenameFilter, Serializable {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,36 +1,76 @@
package com.cloudbees.jenkins.support.impl;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.hasKey;
import static org.hamcrest.Matchers.notNullValue;

import com.cloudbees.jenkins.support.SupportTestUtils;
import hudson.ExtensionList;
import hudson.slaves.DumbSlave;
import java.util.Map;
import org.hamcrest.MatcherAssert;
import org.junit.Rule;
import org.junit.Test;
import org.jvnet.hudson.test.InboundAgentRule;
import org.jvnet.hudson.test.JenkinsRule;

public class SlaveLaunchLogsTest {

@Rule
public JenkinsRule j = new JenkinsRule();

@Rule
public InboundAgentRule inboundAgents = new InboundAgentRule();

@Test
public void onlineOutboundAgent() throws Exception {
var s = j.createOnlineSlave();
assertThat(
"reflects JenkinsRule.createComputerLauncher command & SlaveComputer.setChannel",
SupportTestUtils.invokeComponentToMap(ExtensionList.lookupSingleton(SlaveLaunchLogs.class))
.get("nodes/slave/" + s.getNodeName() + "/launchLogs/slave.log"),
allOf(
notNullValue(),
containsString("-XX:+PrintCommandLineFlags"),
containsString("Remoting version: ")));
}

@Test
public void smokesRoot() throws Exception {
DumbSlave s = j.createSlave();
Map<String, String> output =
SupportTestUtils.invokeComponentToMap(ExtensionList.lookupSingleton(SlaveLaunchLogs.class));
String key = "nodes/slave/" + s.getNodeName() + "/launchLogs/slave.log";
MatcherAssert.assertThat(output, hasKey(key));
public void onlineInboundAgent() throws Exception {
inboundAgents.createAgent(j, "remote");
assertThat(
"reflects DefaultJnlpSlaveReceiver.beforeChannel & SlaveComputer.setChannel",
SupportTestUtils.invokeComponentToMap(ExtensionList.lookupSingleton(SlaveLaunchLogs.class))
.get("nodes/slave/remote/launchLogs/slave.log"),
allOf(
notNullValue(),
containsString("Inbound agent connected from"),
containsString("Communication Protocol: JNLP4-connect")));
}

@Test
public void smokesComputer() throws Exception {
DumbSlave s = j.createSlave();
Map<String, String> output = SupportTestUtils.invokeComponentToMap(
ExtensionList.lookupSingleton(SlaveLaunchLogs.class), s.toComputer());
String key = "nodes/slave/" + s.getNodeName() + "/launchLogs/slave.log";
MatcherAssert.assertThat(output, hasKey(key));
public void component() throws Exception {
var s = j.createSlave();
assertThat(
SupportTestUtils.invokeComponentToMap(
ExtensionList.lookupSingleton(SlaveLaunchLogs.class), s.toComputer()),
hasKey("nodes/slave/" + s.getNodeName() + "/launchLogs/slave.log"));
}

@Test
public void offlineAgent() throws Exception {
var s = j.createOnlineSlave();
var name = s.getNodeName();
s.toComputer().disconnect(null).get();
assertThat(
"still includes something",
SupportTestUtils.invokeComponentToMap(ExtensionList.lookupSingleton(SlaveLaunchLogs.class))
.get("nodes/slave/" + name + "/launchLogs/slave.log"),
allOf(notNullValue(), containsString("Remoting version: ")));
}

// TODO logs from a deleted agent

Check warning on line 71 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLaunchLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: logs from a deleted agent
// TODO logs from multiple launch attempts of same agent

Check warning on line 72 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLaunchLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: logs from multiple launch attempts of same agent
// TODO honor SafeTimerTask.getLogsRoot (if applicable)

Check warning on line 73 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLaunchLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: honor SafeTimerTask.getLogsRoot (if applicable)
// TODO rotation of old or excessively long logs

Check warning on line 74 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLaunchLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: rotation of old or excessively long logs

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/*
* The MIT License
*
* Copyright 2024 CloudBees, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/

package com.cloudbees.jenkins.support.impl;

import org.junit.Rule;
import org.jvnet.hudson.test.JenkinsRule;

public final class SlaveLogsTest {

@Rule
public JenkinsRule j = new JenkinsRule();

// TODO INFO messages from online agent after connection

Check warning on line 35 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: INFO messages from online agent after connection
// TODO messages from online agent prior to connection (if -workDir set)

Check warning on line 36 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: messages from online agent prior to connection (if -workDir set)
// TODO messages from disconnected agent

Check warning on line 37 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: messages from disconnected agent
// TODO messages from deleted agent

Check warning on line 38 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: messages from deleted agent
// TODO messages from reconnected agent

Check warning on line 39 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: messages from reconnected agent
// TODO FINE messages

Check warning on line 40 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: FINE messages
// TODO winsw logs

Check warning on line 41 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: winsw logs
// TODO honor SafeTimerTask.getLogsRoot (if applicable)

Check warning on line 42 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: honor SafeTimerTask.getLogsRoot (if applicable)
// TODO rotation of old or excessively long logs

Check warning on line 43 in src/test/java/com/cloudbees/jenkins/support/impl/SlaveLogsTest.java

View check run for this annotation

ci.jenkins.io / Open Tasks Scanner

TODO

NORMAL: rotation of old or excessively long logs

}
Loading