Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create official Java driver #3930

Closed
deontologician opened this issue Mar 13, 2015 · 100 comments
Closed

Create official Java driver #3930

deontologician opened this issue Mar 13, 2015 · 100 comments
Assignees
Labels
Milestone

Comments

@deontologician
Copy link
Contributor

https://github.com/npiv/rethink-java-driver might be a good jumping off point. It seems both of the 3rd party drivers for Java are abandoned or very out of date

@danielmewes danielmewes added this to the subsequent milestone Mar 18, 2015
@danielmewes danielmewes modified the milestones: 2.1, subsequent Apr 8, 2015
@danielmewes
Copy link
Member

I think we should try to get a beta release out by 2.1, especially since we might abandon protocol buffer support with that release and @npiv's driver is still using it.

Later we'll need to integrate it into our polyglot testing framework.

Another possible starting point for a driver is @dkhenry's Java driver. I've already ported it to the JSON protocol for benchmarking purposes in https://github.com/danielmewes/rethinkjava , but in contrast to @npiv's driver it's missing support for lambdas.

@deontologician
Copy link
Contributor Author

Most likely the serialization stuff is easier to port from one to the other. There are some language design issues to consider as well though, I think the default should return a cursor, rather than needing runForCursor. So in any case, there would be a lot of fiddling to get it into shape

@kclay
Copy link

kclay commented Apr 21, 2015

I've been meaning to add more javaness to my Scala driver( https://github.com/kclay/rethink-scala) since it enforces type safety and provides methods for casting, blocking and async modes ,jackson for json,etc. On @deontologician note about the need for runForCursor. The driver is built in a way that its able to determine if the result should be a single value or an cursor. Pretty sure I can refactor some of this to work with java since some parts relying strongly on Scala implicits. Just a thought.

@stephanos
Copy link

@deontologician
Will the new driver support Java 7?

@stephanos
Copy link

@danielmewes
I can't even get your fork to compile! After some fixing I am stuck with Error:(296, 37) java: incompatible types: long cannot be converted to com.google.protobuf.CodedInputStream (RqlConnection.java) @ https://github.com/stephanos/rethinkjava

@danielmewes
Copy link
Member

@stephanos I saw you did some patches in your branch, did you get it to compile eventually? I'm not sure why it's not compiling (though I'm not surprised, I've only ever used it on one type of system for a single purpose). My guess would be an incompatible protocol buffer library or compiler.

The official driver will not be based on that code base by the way.

@stephanos
Copy link

@danielmewes no, I didn't :( -will have to wait for the official driver.

@danielmewes
Copy link
Member

@stephanos another driver you can try until then is https://github.com/dropbox/rethink-java-driver

@deontologician
Copy link
Contributor Author

@stephanos I'm building it with Java 8, and we will probably release the beta version tested only with Java 8, but supporting Java 7 shouldn't be too hard and I think we should support it as part of release version

@deontologician
Copy link
Contributor Author

Actually @stephanos, what is your opinion on the Java 7 EOL announcement?

I assume there are lots of people still on Java 7, but I can't imagine everyone is able to switch right away. So as someone using Java 7, I'm curious whether upgrading is on your roadmap or what the feeling is towards Java 8 from professional Java developers in general

@stephanos
Copy link

@deontologician Well, I'd (almost) give a kidney to be able to use Java 8 right now. Problem is that we run on App Engine which only supports Java 7 - and it looks like this won't change, ever. The new option is ManagedVM, but it's only in beta status, has no SLA and is super slow to develop for (it is based on boot2docker). So we're stuck on Java 7 :(

@deontologician
Copy link
Contributor Author

@stephanos thanks, that's really useful info

@iboumedien
Copy link

Thanks

@scottwleonard
Copy link

@deontologician I wanted to add my $.02 to your question about Java 8. We have been using Java 8 for awhile now, and there are a ton of useful features. streams are very well implemented from a syntax perspective, and they have allowed us to write our code in a more functional way. We didn't have any issues converting our codebase over from Java 7.

@deontologician
Copy link
Contributor Author

Thanks @scottwleonard, that's helpful. I'm a fan of Java 8, I think (despite being Oracle) that they did a good job integrating functional language features into the existing language in a way that is really nice to work with.

@gmethvin
Copy link

I'd also like to see good Scala driver support but once the low level stuff is officially supported in a Java driver I think it'd be fairly easy to write a small Scala layer on top of it with a nice DSL and maybe some additional type safety.

For the Java driver, ideally the serialization framework should be pluggable, and it'd be good if cursors implemented reactive streams so they'd easily interoperate with Play iteratees and Akka streams.

@danielmewes
Copy link
Member

@gmethvin Thanks for the suggestions.

I think we'll first make sure that we ship a Java driver on par with our other official drivers and with a comparable interface that will be based on synchronous cursors.
Once we have that working reliably, I suspect that adding a reactive stream API is going to be fairly easy from what I can tell (I had only a very brief look at https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.0/README.md , so there might be some complications that I missed).

@deontologician do you have additional comments on that?

@gmethvin What is your use case for a pluggable serialization framework? Are you talking about the JSON encoder/decoder, the connection logic, or some higher-level API?

@deontologician
Copy link
Contributor Author

I don't have much to add except that it will take me a bit of research to find out what the best approach is for async etc. The Java ecosystem is really big, so it might make sense to do something like the python driver where the async library is swappable.

@gmethvin
Copy link

@danielmewes I'm talking about the JSON serialization. It would be nice if I'm already using Jackson, Gson, Play Json, etc., to easily write an adapter to serialize/deserialize my objects.

@danielcompton
Copy link
Contributor

I maintain the Clojure RethinkDB library, and would be very interested in using an official RethinkDB Java driver for connection management and cursors, and wrapping/adapting this to be idiomatic in Clojure.

@gmethvin
Copy link

Also, the type system should try to prevent you from doing things that are obviously wrong. It shouldn't be possible to have a runtime error like Indexed order_by can only be performed on a TABLE or TABLE_SLICE since the type system prevents you from calling the method.

@danielmewes
Copy link
Member

I'm quite sceptical about putting anything but trivial type constraints into the driver.
It adds additional code and increases the maintenance burden, since we would need to update the driver when adding new type signatures to ReQL.

@igorlukanin
Copy link
Contributor

@mattias800 There're also tests with usage examples for the POJO stuff mentioned by @deontologician.

@deontologician I bet @tyth would really like if we make r.table(...)'s signature not just r.table(Object tableName), but provide a more strongly typed r.table(ReqlExpr tableNameExpr) among with probably more frequently used r.table(String tableName) which would just wrap the string with the appropriate ReqlExpr.

I think there's no harm in doing this in means of maintainability, and it could be done anytime after 2.2 (kind of 1.0 for Java driver) is released.

@mattias800
Copy link

Nice! Gonna try it out tomorrow!

@pires
Copy link
Contributor

pires commented Nov 9, 2015

@crockpotveggies and everyone else, just be aware that the artifact available on OSS Sonatype atm is dated Oct 26th and is outdated when compared to current code. Better build manually and install it locally.

@dklotz
Copy link

dklotz commented Nov 10, 2015

@mattias800

Having it return instances of your own classes would be a great feature though.

This (in essence: ODM / object-document-mapping) could be compared to e.g. what Morphia is doing for MongoDB on top of their normal java driver and I would strongly suggest to have it in a library separate from the basic driver. Definitely needed, but definitely a different beast than the basic DB driver.

@danielmewes danielmewes modified the milestones: 2.2, 2.2-polish, 2.2.x Nov 11, 2015
@deontologician
Copy link
Contributor Author

OK guys, I uploaded a new snapshot. We've also merged the driver to next (the main development branch) so things are coming along!

@crockpotveggies
Copy link
Contributor

Nice work! Unless someone found a fix, I think compatibility with Scala case class is currently broken, so not sure I can test the new version. It will require some thought around reflection since case classes are structured a little differently.

@pires
Copy link
Contributor

pires commented Nov 21, 2015

@deontologician 👏

@radicaled
Copy link

I've noticed that the POJO mapping code intentionally ignores fields that are inherited.

So, simple hierarchies like this fail:

class Model {
  // too lazy to write out getter / setter, pretend I did
  public String id;
}

// id will always be null if you pass this class, like so: `...run(connection, Person.class)`
class Person extends Model {
  // too lazy to write out getter / setter, pretend I did
   public String name;
}

Is there any particular reason why the POJO mapper does this?

@pires
Copy link
Contributor

pires commented Nov 21, 2015

I've also noticed it doesn't support Enum.

@kclay
Copy link

kclay commented Nov 21, 2015

I would suggest  looking into Jackson for POJO support there are a lot if corner cases as well if the intended use was to also be able to support Scala case classes as well. I ended up using Jackson for the rethink-scala library and it worked out pretty well.
On Nov 21, 2015 6:12 AM, Paulo Pires notifications@github.com wrote:I've also noticed it doesn't support Enum.
On Nov 21, 2015 11:33, "Arron Washington" notifications@github.com wrote:

I've noticed that the POJO mapping code intentionally ignores fields that
are inherited.

So, simple hierarchies like this fail:

class Model {
public String id;
}
// id will always be null if you pass this class, like so: ...run(connection, Person.class)class Person extends Model {
public String name;
}

Is there any particular reason why the POJO mapper does this?


Reply to this email directly or view it on GitHub
#3930 (comment)
.

—Reply to this email directly or view it on GitHub.

@deontologician
Copy link
Contributor Author

@kclay you may be right. I opened #5128 to look at what would be required

@berndwarmuth
Copy link

When I try to call insert in parallel I noticed that the driver crashes with a "java.lang.OutOfMemoryError" Exception. This happens as soon as more than one calls to insert are fired in parallel. Any suggestions how to fix this?

Here is the stacktrace:

java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.rethinkdb.net.Util.leByteBuffer(Util.java:25)
at com.rethinkdb.net.SocketWrapper.recvall(SocketWrapper.java:100)
at com.rethinkdb.net.ConnectionInstance.readResponse(ConnectionInstance.java:74)
at com.rethinkdb.net.ConnectionInstance.readResponse(ConnectionInstance.java:58)
at com.rethinkdb.net.Connection.runQuery(Connection.java:156)
at com.rethinkdb.net.Connection.run(Connection.java:204)
at com.rethinkdb.ast.ReqlAst.run(ReqlAst.java:66)

kind regards,

@bchavez
Copy link
Contributor

bchavez commented Dec 2, 2015

When I ported over the Java driver code to C#, I noticed some internal data structures were not thread-safe in C# land. All the threading issues are resolved C# driver & documented in this internal design note:

bchavez/RethinkDb.Driver#15

Although, this design note is for C#, it might help identify thread-sensitive spots in the Java driver since both drivers share the same architecture.

@danielmewes
Copy link
Member

@triggetry I think @bchavez is right that this is likely a threading problem. For now I recommend using one connection per thread.

@danielmewes
Copy link
Member

Opened #5166 to add thread-safety to the Java driver.

@a1russell
Copy link

It's released! Yay!

https://www.rethinkdb.com/docs/install-drivers/
http://rethinkdb.com/blog/official-java-driver/

@deontologician
Copy link
Contributor Author

This is fixed now.

@mglukhovsky
Copy link
Member

Great work, @deontologician! ☕

@crockpotveggies
Copy link
Contributor

Just wanted to say nice work to everyone who got this developed. Not sure if Scala support made it for the release, however it's very meaningful to the JVM community :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests