解决dinky0.7.4在Flink on Yarn模式中报TimelineUtils类NoClassDefFoundError的问题

0x00 生态版本

flink	dinky	hdfs
1.17.1	0.7.4	3.1.1

0x01 报错信息

在进行Flink on yarn 集群配置的时候，配置好点击测试，dinky的UI界面会报两个错，这点dinky做的比较好，错误提示很明显，就算不翻日志也能知道问题：

第一个错误：
java.lang.ClassNotFoundException: javax.ws.rs.ext.MessageBodyReader

第二个错误：
Handler dispatch failed; nested exception is java\.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils

报错截图如下：

完整的配置如下：

错误细节信息可以详见dinky的日志：

Caused by: java.lang.ClassNotFoundException: javax.ws.rs.ext.MessageBodyReader
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_202]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_202]
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_202]
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_202]
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_202]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_202]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_202]
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_202]
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_202]
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_202]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_202]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_202]
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_202]
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_202]
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_202]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_202]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_202]
        at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_202]
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_202]
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_202]
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_202]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_202]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_202]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_202]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_202]
        at org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:58) ~

[dlink] 2023-10-24 10:49:29 CST ERROR com.dlink.exception.WebExceptionHandler 97 unknownException - ERROR: org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1086) ~[spring-webmvc-5.3.23.jar:5.3.23]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964) ~[spring-webmvc-5.3.23.jar:5.3.23]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~

0x02 解决方法

下载 flink-shaded-hadoop-3-uber-3.1.1.7.2.9.0-173-9.0.jar、javax.ws.rs-api-2.0.1.jar文件，并复制yarn安装目录中的hadoop-mapreduce-client-core-3.1.1.jar到$DLINK_HOME/plugins文件夹下。

最后，看完整的包。在 $DLINK_HOME/plugins 下放置如下包：

[hadoop@hadoop003 plugins]$ ll
总用量 59948
drwxr-xr-x 3 hadoop hadoop     4096 10月 16 17:51 flink1.17
-rwxr-xr-x 1 hadoop hadoop 59604787 10月 16 17:33 flink-shaded-hadoop-3-uber-3.1.1.7.2.9.0-173-9.0.jar
-rwxr-xr-x 1 hadoop hadoop  1654881 10月 16 17:33 hadoop-mapreduce-client-core-3.1.1.jar
-rwxr-xr-x 1 hadoop hadoop   115534 10月 16 17:33 javax.ws.rs-api-2.0.1.jar

同时把$FLINK_HOME/lib中的文件复制到 $DLINK_HOME/plugins/flink1.17中：

-rwxr-xr-x 1 hadoop hadoop     53820 10月 16 17:51 commons-cli-1.5.0.jar
drwxr-xr-x 2 hadoop hadoop        83 10月 16 17:33 dinky
-rwxr-xr-x 1 hadoop hadoop     21154 10月 16 17:51 dinky-catalog-mysql-1.17-1.0.0-SNAPSHOT.jar
-rwxr-xr-x 1 hadoop hadoop      4617 10月 16 17:51 failureaccess-1.0.1.jar
-rwxr-xr-x 1 hadoop hadoop    196491 10月 16 17:51 flink-cep-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop    542620 10月 16 17:51 flink-connector-files-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop    264302 10月 16 17:51 flink-connector-jdbc-3.1.0-1.17.jar
-rwxr-xr-x 1 hadoop hadoop    403255 10月 16 17:51 flink-connector-kafka-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop    102472 10月 16 17:51 flink-csv-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop 135975541 10月 16 17:51 flink-dist-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop    180248 10月 16 17:51 flink-json-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop  21043319 10月 16 17:51 flink-scala_2.12-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop   3171088 10月 16 17:51 flink-shaded-guava-31.1-jre-17.0.jar
-rwxr-xr-x 1 hadoop hadoop  51337510 10月 16 17:51 flink-sql-connector-hive-3.1.3_2.12-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop   5563748 10月 16 17:51 flink-sql-connector-kafka-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop  18897491 10月 16 17:51 flink-sql-connector-mongodb-cdc-2.4.1.jar
-rwxr-xr-x 1 hadoop hadoop  23600947 10月 16 17:51 flink-sql-connector-mysql-cdc-2.4.1.jar
-rwxr-xr-x 1 hadoop hadoop    731203 10月 16 17:51 flink-sql-connector-rabbitmq-1.17-20230110.003509-121.jar
-rwxr-xr-x 1 hadoop hadoop  74942397 10月 16 17:51 flink-sql-connector-tidb-cdc-2.4.1.jar
-rwxr-xr-x 1 hadoop hadoop  15407424 10月 16 17:51 flink-table-api-java-uber-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop  21331589 10月 16 17:51 flink-table-planner_2.12-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop   3146210 10月 16 17:51 flink-table-runtime-1.17.1.jar
-rwxr-xr-x 1 hadoop hadoop   1654881 10月 16 17:51 hadoop-mapreduce-client-core-3.1.1.jar
-rwxr-xr-x 1 hadoop hadoop    208006 10月 16 17:51 log4j-1.2-api-2.17.1.jar
-rwxr-xr-x 1 hadoop hadoop    301872 10月 16 17:51 log4j-api-2.17.1.jar
-rwxr-xr-x 1 hadoop hadoop   1790452 10月 16 17:51 log4j-core-2.17.1.jar
-rwxr-xr-x 1 hadoop hadoop     24279 10月 16 17:51 log4j-slf4j-impl-2.17.1.jar
-rwxr-xr-x 1 hadoop hadoop  36096449 10月 16 17:51 paimon-flink-1.17-0.6-20230916.001857-12.jar
-rwxr-xr-x 1 hadoop hadoop     13071 10月 16 17:51 paimon-flink-action-0.6-20230916.001857-12.jar
-rwxr-xr-x 1 hadoop hadoop  35359604 10月 16 17:51 paimon-hive-connector-3.1-0.6-20230916.001857-10.jar
-rwxr-xr-x 1 hadoop hadoop  35683112 10月 16 17:51 paimon-spark-3.3-0.6-20230916.001857-10.jar
-rwxr-xr-x 1 hadoop hadoop  42627639 10月 16 17:51 paimon-trino-422-0.6-20230916.000432-10.jar

附：

• flink-shaded-hadoop-3-uber-3.1.1.7.2.9.0-173-9.0.jar的下载地址：https://repository.cloudera.com/repository/cloudera-repos/org/apache/flink/flink-shaded-hadoop-3-uber/3.1.1.7.2.9.0-173-9.0/flink-shaded-hadoop-3-uber-3.1.1.7.2.9.0-173-9.0.jar
• javax.ws.rs-api-2.0.1.jar的下载地址：https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar