Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: dynamically load BigtableAdmin and BigtableAsyncAdmin #3341

Merged
merged 10 commits into from
Nov 29, 2021

Conversation

mutianf
Copy link
Contributor

@mutianf mutianf commented Nov 23, 2021

When upgrading hbase from 1.4.x to 1.7.x, hbase added a method in the interface which depends on ServerType class that only exists after 1.7.0 (https://github.com/googleapis/java-bigtable-hbase/pull/3270/files#diff-5ec033f24f0ec19af948a12e09d84d70f8d9342c098cdca7cb1f6c2f71fe399dR334). This breaks hbase shell that's running with an older version of hbase because hbase shell (which is written in ruby) will try to load all the methods and won't be able to load the ServerType class because it doesn't exist in an older version of hbase. The error message looks like:

ERROR: org.apache.hadoop.hbase.client.ServerType
Backtrace: java.net.URLClassLoader.findClass(URLClassLoader.java:382)
           java.lang.ClassLoader.loadClass(ClassLoader.java:418)
           sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
           java.lang.ClassLoader.loadClass(ClassLoader.java:351)
           java.lang.Class.getDeclaredMethods0(Native Method)
           java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
           java.lang.Class.getDeclaredMethods(Class.java:1975)
           org.jruby.javasupport.JavaClass.getMethods(JavaClass.java:2110)
           org.jruby.javasupport.JavaClass.setupClassMethods(JavaClass.java:955)
           org.jruby.javasupport.JavaClass.access$700(JavaClass.java:99)
           org.jruby.javasupport.JavaClass$ClassInitializer.initialize(JavaClass.java:650)
           org.jruby.javasupport.JavaClass.setupProxy(JavaClass.java:689)
           org.jruby.javasupport.Java.createProxyClass(Java.java:526)
           org.jruby.javasupport.Java.getProxyClass(Java.java:455)
           org.jruby.javasupport.Java.getInstance(Java.java:364)
           org.jruby.javasupport.JavaUtil.convertJavaToUsableRubyObject(JavaUtil.java:166)
           org.jruby.javasupport.JavaMethod.convertReturn(JavaMethod.java:512)
           org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:436)
           org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:299)
           org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:50)
           org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
           org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)

This pr tries to workaround this issue by dynamically loading BigtableAdmin and BigtableAsyncAdmin classes so the backward incompatible methods won't be accessed unless the methods are called.

@mutianf mutianf requested review from a team as code owners November 23, 2021 18:47
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Nov 23, 2021
@product-auto-label product-auto-label bot added the api: bigtable Issues related to the googleapis/java-bigtable-hbase API. label Nov 23, 2021
new AbstractBigtableAdmin.UnsupportedOperationsHandler()))
.make()
.load(BigtableAdmin.class.getClassLoader())
.getLoaded()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think we should be defining a new class for every invocation. Instead I think it would be better to move the class definition to BigtableAdmin, something along the lines of:

abstract class BIgtableAdmin {
private static Class<? extends BigtableAdmin> classDefinition = null;
private static synchronized Class<? extends BigtableAdmin> getSubclass() {
if (classDefinition == null) {
// byte buddy...
}
return classDefinition;
}
public static Admin createInstance(Connection connection) {
return getSubclass().newInstance()...;
}

asyncAdmin = new BigtableAsyncAdmin(connection);
try {
asyncAdmin = BigtableAsyncAdmin.createInstance(connection);
} catch (Exception e) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think the try catch necessary here


private static Class<? extends BigtableAdmin> adminClass = null;

private static synchronized Class<? extends BigtableAdmin> getSubclass() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add docs explaining what this is doing and why its necessary

return new BigtableAsyncAdmin(BigtableAsyncConnection.this);
} catch (IOException e) {
return BigtableAsyncAdmin.createInstance(BigtableAsyncConnection.this);
} catch (Exception e) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is try catch necessary here?

Copy link
Collaborator

@igorbernstein2 igorbernstein2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mutianf mutianf added the owlbot:run Add this label to trigger the Owlbot post processor. label Nov 29, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Nov 29, 2021
@mutianf mutianf merged commit 18b2e18 into googleapis:main Nov 29, 2021
@mutianf mutianf deleted the hbase branch November 29, 2021 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/java-bigtable-hbase API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants