[FLINK-38912] Upgrade Netty to 4.2.6.Final

[mateczagany]

What is the purpose of the change

Upgrade Netty to 4.2.6.Final to match the version of flink-shaded. This is meant to compliment PR #27407.

Brief change log

• Upgrade netty-bom in parent pom
• In flink-rpc-akka, filter out all Netty .class files to ensure that they don't conflict with flink-shaded ones
• Pin Netty version to 4.1.100.Final in flink-python until Apache Arrow is upgraded
• Remove unnecessary Netty version override in flink-kubernetes

Verifying this change

CI should pass

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): yes
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[flinkbot]

CI report:

• 3d3d871 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[ferenc-csaky]

Thanks for taking care of this. Logic LGTM, just left some questions before approve.

[ferenc-csaky]

Can we also include the current Arrow version here so it will be more obvious in the future which Arrow version was the cause for this without browsing mvnrepo and its dependency matrix? :)

[mateczagany]

Yes, I've added a comment. Unfortunately there are no currently released versions of Apache Arrow that's built against Netty 4.2.x, the upgrade was done 3 weeks ago: apache/arrow-java@7d4cf21

[ferenc-csaky]

I mean to include the arrow version explicitly to the comment that will resolve the current situation. Something like:

"... It should be removed or updated once Apache Arrow is upgraded to version x.y.z+."

[mateczagany]

Sorry, I've left this comment and updated it locally, just did not push until I've finished looking into your other comment. I've pushed the update now.

[ferenc-csaky]

Was this wrong before, or is it something new to handle?

[mateczagany]

The issue here was that some Netty classes required by Pekko were not properly shaded as netty-codec was reworked in the latest Netty release. Not sure why, but depending on netty-all will result in a lot of classes, such as LengthFieldPrepender not being included in the shading step.

My idea here was to not package any shaded Netty dependencies from this module, and instead only use the already shaded classes from flink-shaded-netty. This would be inconsistent compared to how this module worked before, so I've gone with another fix:

• Depend only on Netty dependencies that are required by Pekko (this reduces the amount of dependencies and fixes the shading issue mentioned)
• Update licenses, as they were quite outdated.

I think this makes the most sense, as we don't need ALL of Netty dependencies coming from netty-all, as we can be quite certain of what Pekko actually needs by looking at their dependencies.