0x00 flink版本

flink发行版：1.17.1

0x01 报错信息

运行Flink Yarn模式的会话部署模式时，程序报错，具体信息如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
[hadoop@hadoop001 flink]$ bin/yarn-session.sh -nm test -d
... 
...
[hadoop@hadoop001 flink]$ bin/flink run -s test -c org.apache.flink.examples.java.wordcount.WordCount examples/batch/WordCount.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/usdp-srv/srv/udp/2.0.0.0/flink/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/usdp-srv/srv/udp/2.0.0.0/yarn/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2023-10-19 09:32:49,402 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                [] - Found Yarn properties file under /tmp/.yarn-properties-hadoop.
2023-10-19 09:32:49,402 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                [] - Found Yarn properties file under /tmp/.yarn-properties-hadoop.
java.lang.NoSuchMethodError: org.apache.commons.cli.CommandLine.hasOption(Lorg/apache/commons/cli/Option;)Z
        at org.apache.flink.client.cli.CliFrontendParser.createSavepointRestoreSettings(CliFrontendParser.java:631)
        at org.apache.flink.client.cli.ProgramOptions.<init>(ProgramOptions.java:119)
        at org.apache.flink.client.cli.ProgramOptions.create(ProgramOptions.java:192)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:235)
        at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1095)
        at org.apache.flink.client.cli.CliFrontend.lambda$mainInternal$9(CliFrontend.java:1189)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.client.cli.CliFrontend.mainInternal(CliFrontend.java:1189)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1157)
[hadoop@hadoop001 flink]$

0x02 解决方式

安装flink时，在lib目录引入了 commons-cli-1.4.jar ，通过命令我们查看到该包中有org.apache.commons.cli.CommandLine这个类：

1
2
3
4
[hadoop@hadoop002 lib]$ jar -tvf commons-cli-1.4.jar | grep CommandLine
   423 Thu Mar 09 14:01:38 CST 2017 org/apache/commons/cli/CommandLineParser.class
  1078 Thu Mar 09 14:01:38 CST 2017 org/apache/commons/cli/CommandLine$Builder.class
  5675 Thu Mar 09 14:01:38 CST 2017 org/apache/commons/cli/CommandLine.class

具体有没有org.apache.commons.cli.CommandLine.hasOption(Lorg/apache/commons/cli/Option;)Z方法，需要进一步验证确认：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
[hadoop@hadoop002 flink]$ mkdir test
[hadoop@hadoop002 flink]$ mv commons-cli-1.4.jar test/
[hadoop@hadoop002 flink]$ cd test/
[hadoop@hadoop002 test]$ ls
commons-cli-1.4.jar
[hadoop@hadoop002 test]$ unzip commons-cli-1.4.jar 
Archive:  commons-cli-1.4.jar
  inflating: META-INF/MANIFEST.MF    
   creating: org/
   creating: org/apache/
   creating: org/apache/commons/
   creating: org/apache/commons/cli/
   creating: META-INF/maven/
   creating: META-INF/maven/commons-cli/
   creating: META-INF/maven/commons-cli/commons-cli/
  inflating: META-INF/LICENSE.txt    
  inflating: org/apache/commons/cli/GnuParser.class  
  inflating: org/apache/commons/cli/HelpFormatter$1.class  
  inflating: org/apache/commons/cli/MissingArgumentException.class  
  inflating: org/apache/commons/cli/Option$Builder.class  
  inflating: org/apache/commons/cli/Options.class  
  inflating: org/apache/commons/cli/PatternOptionBuilder.class  
  inflating: org/apache/commons/cli/Util.class  
  inflating: META-INF/NOTICE.txt     
  inflating: org/apache/commons/cli/BasicParser.class  
  inflating: org/apache/commons/cli/CommandLineParser.class  
  inflating: org/apache/commons/cli/HelpFormatter.class  
  inflating: org/apache/commons/cli/OptionGroup.class  
  inflating: org/apache/commons/cli/Parser.class  
  inflating: org/apache/commons/cli/UnrecognizedOptionException.class  
  inflating: org/apache/commons/cli/AlreadySelectedException.class  
  inflating: org/apache/commons/cli/CommandLine$Builder.class  
  inflating: org/apache/commons/cli/CommandLine.class  
  inflating: org/apache/commons/cli/HelpFormatter$OptionComparator.class  
  inflating: org/apache/commons/cli/Option$1.class  
  inflating: org/apache/commons/cli/OptionValidator.class  
  inflating: org/apache/commons/cli/PosixParser.class  
  inflating: META-INF/maven/commons-cli/commons-cli/pom.xml  
  inflating: org/apache/commons/cli/AmbiguousOptionException.class  
  inflating: org/apache/commons/cli/DefaultParser.class  
  inflating: org/apache/commons/cli/MissingOptionException.class  
  inflating: org/apache/commons/cli/Option.class  
  inflating: org/apache/commons/cli/OptionBuilder.class  
  inflating: org/apache/commons/cli/ParseException.class  
  inflating: org/apache/commons/cli/TypeHandler.class  
  inflating: META-INF/maven/commons-cli/commons-cli/pom.properties  
[hadoop@hadoop002 test]$ 
[hadoop@hadoop002 test]$ ls
commons-cli-1.4.jar  META-INF  org
[hadoop@hadoop002 test]$ 
[hadoop@hadoop002 test]$ 
[hadoop@hadoop002 test]$ javap -p org/apache/commons/cli/CommandLine.class 
Compiled from "CommandLine.java"
public class org.apache.commons.cli.CommandLine implements java.io.Serializable {
  private static final long serialVersionUID;
  private final java.util.List<java.lang.String> args;
  private final java.util.List<org.apache.commons.cli.Option> options;
  protected org.apache.commons.cli.CommandLine();
  public boolean hasOption(java.lang.String);
  public boolean hasOption(char);
  public java.lang.Object getOptionObject(java.lang.String);
  public java.lang.Object getParsedOptionValue(java.lang.String) throws org.apache.commons.cli.ParseException;
  public java.lang.Object getOptionObject(char);
  public java.lang.String getOptionValue(java.lang.String);
  public java.lang.String getOptionValue(char);
  public java.lang.String[] getOptionValues(java.lang.String);
  private org.apache.commons.cli.Option resolveOption(java.lang.String);
  public java.lang.String[] getOptionValues(char);
  public java.lang.String getOptionValue(java.lang.String, java.lang.String);
  public java.lang.String getOptionValue(char, java.lang.String);
  public java.util.Properties getOptionProperties(java.lang.String);
  public java.lang.String[] getArgs();
  public java.util.List<java.lang.String> getArgList();
  protected void addArg(java.lang.String);
  protected void addOption(org.apache.commons.cli.Option);
  public java.util.Iterator<org.apache.commons.cli.Option> iterator();
  public org.apache.commons.cli.Option[] getOptions();
}

注意看，只有两个同名不同参数的方法：

1
2
public boolean hasOption(java.lang.String);
public boolean hasOption(char);

并没有一个参数类型为org/apache/commons/cli/Option的方法，因此 commons-cli-1.4.jar不满足flink1.17.1版本。

我们找到一个最新的包： commons-cli-1.5.0，下载后继续使用上面方法验证，找到了与报错信息一致的方法：

1
2
3
4
5
[hadoop@hadoop002 test]$ unzip commons-cli-1.5.0.jar 
[hadoop@hadoop002 test]$ javap -p org/apache/commons/cli/CommandLine.class  | grep hasOption
  public boolean hasOption(char);
  public boolean hasOption(org.apache.commons.cli.Option);
  public boolean hasOption(java.lang.String);

因此我们flink1.17.1需要commons-cli-1.5.0.jar 的包。