Add examples of usage of non OSGi clients/remote service consumers (java and other grpc-supported languages)

[scottslewis]

The GRPC distribution provider allows for support of non-OSGi remote service consumers (e.g. java applications) as well use of non-java clients/consumers (i.e. with the multi-language support provided by grpc see

https://grpc.io/docs/languages/

This means that no additional work needs to be done (other than compiling the proto file defined for the service in chosen language) to use non-OSGi java and non-Java consumers for a remote service exported by the grpc distribution provider in OSGi server environments (e.g. Karaf).

It would be helpful to add examples of doing this for the HealthCheck example remote service:

1. Java application example (no OSGi)
2. Other language examples: C#, C++, Python, Node, Go, etc.... from https://grpc.io/docs/languages

[scottslewis]

Added Java application for accessing HealthCheck service via health api project: https://github.com/ECF/grpc-RemoteServicesProvider/tree/master/examples/org.eclipse.ecf.examples.provider.grpc.health.api project dependency in this new (java app) project:

https://github.com/ECF/grpc-RemoteServicesProvider/tree/master/examples/health.consumer.javaapp

[eum2o]

Hi Scott. Thanks for the mention in the copyright notice. I really wouldn't have demanded this for this little code I wrote but I won't complain either ;)

Review

It provides an executable example how to consume an OSGi remote service with a non-OSGi-client / plain Java project and thus fulfills its purpose. The code is simple and easy to understand. ✅

What I would update/fix

When you import the new project into Eclipse, the compiler creates two error markers due to the missing src-folders src/main/resources and src/test/resources (you probably either need to remove those two folders from the classpath or add placeholder dummy files so that git has something to add/commit).

Worthy to discuss/At least I want to mention this thought

I would think about removing the dependency to org.eclipse.ecf.examples.provider.grpc.health.api and instead add a copy of the .proto file to the project and generate the sources. The dependency completely makes sense: When you have a Java project providing the API, why not use it. But I imagine the following advantages in replacing the dependency with a .proto:

• It proves that you need nothing but the proto file to consume the service (you don't have to verify that the .api project contains no other API than the generated proto sources)
• Consistency to future examples written in other languages where you cannot add a dependency to the .api project. E.g. when you have examples for Python, Go, etc. in the future, they probably will have a .proto file too - and when you look into the Java example one might wonder why there's no .proto file in this particular case)
• If you install the health check example as described in the readme.md e.g. into Karaf and you want to consume it using the example project, you'd only have to checkout/import the plain Java project into your Eclipse workspace. No dependencies would be required. Currently, you'd have to import at least the .api project into your workspace, too. And this maybe leads to error markers because dependencies (e.g. org.eclipse.ecf.provider.grpc) cannot be found. You first had to find out which updatesite/feature to install to satisfy this dependency, etc.

(But it would also be completely fine to keep the dependency from my perspective - just mentioning some thoughts).

---

Out of scope: When I installed the health check example I noticed it isn't working as described in the readme.md using feature:install -v ecf-rs-examples-grpc-healthcheck-impl. It complains about the unresolveable bundle ecf-rs-examples-grpc-healthcheck-api/1.2.0. I assume that the version attribute of a feature node is no minimum version but "match exactly". I worked around it by installing the health check api/impl bundles directly.

[scottslewis]

Hi Scott. Thanks for the mention in the copyright notice. I really wouldn't have demanded this for this little code I wrote but I won't complain either ;)

You are welcome. This is a contribution-based OS project and all contributions will be acknowledged. In return I ask for/hope for some promotion of the work in ways available (demos/video, blog, other examples contributions...this issue, articles, etc). FWIW, I'm particularly keen on promoting and continuing to improve the grpc distribution provider because...along with other things like bndtools tooling (with automatic execution of protoc on every .proto source change), my intention to distribute the protofile Eclipse editor plugin and do other things to make grpc + OSGi usable in as many use/debugging/testing cases as possible.

Review

It provides an executable example how to consume an OSGi remote service with a non-OSGi-client / plain Java project and thus fulfills its purpose. The code is simple and easy to understand. ✅

What I would update/fix

When you import the new project into Eclipse, the compiler creates two error markers due to the missing src-folders src/main/resources and src/test/resources (you probably either need to remove those two folders from the classpath or add placeholder dummy files so that git has something to add/commit).

I've fixed this in the project's .classpath.

Worthy to discuss/At least I want to mention this thought

I would think about removing the dependency to org.eclipse.ecf.examples.provider.grpc.health.api and instead add a copy of the .proto file to the project and generate the sources. The dependency completely makes sense: When you have a Java project providing the API, why not use it. But I imagine the following advantages in replacing the dependency with a .proto:

* It proves that you need nothing but the proto file to consume the service (you don't have to verify that the .api project contains no other API than the generated proto sources)

* Consistency to future examples written in other languages where you cannot add a dependency to the .api project. E.g. when you have examples for Python, Go, etc. in the future, they probably will have a .proto file too - and when you look into the Java example one might wonder why there's no .proto file in this particular case)

* If you install the health check example as described in the `readme.md` e.g. into Karaf and you want to consume it using the example project, you'd only have to checkout/import the plain Java project into your Eclipse workspace. No dependencies would be required. Currently, you'd have to import at least the .api project into your workspace, too. And this maybe leads to error markers because dependencies (e.g. `org.eclipse.ecf.provider.grpc`) cannot be found. You first had to find out which updatesite/feature to install to satisfy this dependency, etc.

(But it would also be completely fine to keep the dependency from my perspective - just mentioning some thoughts).

I agree with all your arguments but I think I'm keeping things as they are for the moment...here are some reasons why:

The grpc-java project has plenty of "plain 'ol java" examples where generating the API code (via maven plugin), and having a server and consumer that export/import this generated api is what is done. My intention/desire with the grpc distribution provider is to support all of: osgi consumer -> java server, java consumer -> osgi server, and osgi consumer -> osgi server. Given this I think it would be better to have one project for the api (in this case o.e.e.provider.grpc.health.api) and figure out ways to have the build figure out how to deal with it. I don't want to promote the notion of more than one copy of the proto source file (i.e. one copy on server, one copy in consumer) because that's going to be difficult to maintain.

Yes I do want to make it as easy as possible for people to develop both an osgi consumer and java consumer of a remote service...and there's probably some better trickery that I could do in the new java consumer maven pom.xml to use the o.e.e.provider.grpc.health.api. Actually, maybe the answer is to make this provider and examples available via maven central.

BTW another thing that's on my mind is that with the bndtools code generation (which allows one to use any external app...i.e. protoc + plugins) and runs every time a change is detected in the proto file. The model for java|osgi consumer -> osgi server is as simple as:

1. Create the proto file
2. Create a bndtools/bundle project, put this proto file in the project and have the bndtools protoc code generation run (via bnd.bnd file addition)...this will gen all java classes, expose to other eclipse projects (bndtools or not) in workspace and expose via bnd-created jar file...all with no programmer intervention (or necessity to create pom.xml by hand).
3. Write servers and consumers (osgi or not) using this one copy of the proto file and generated classes from 2.

This will hopefully make it much easier for people to create apis from protofiles (with bndtools and auto protoc code gen) and then create, test, debug, and build and deploy osgi, java, or other language (might try python next with pydev) while still being able to modify (and regen) proto-define service apis.

In any event...I will be doing more on this (create, dev, debug, test, deploy, iterate) for these osgi + java and other lang use cases.

Out of scope: When I installed the health check example I noticed it isn't working as described in the readme.md using feature:install -v ecf-rs-examples-grpc-healthcheck-impl. It complains about the unresolveable bundle ecf-rs-examples-grpc-healthcheck-api/1.2.0. I assume that the version attribute of a feature node is no minimum version but "match exactly". I worked around it by installing the health check api/impl bundles directly.

Ok, thanks. This is just a typo/my mistake I think...but I will fix it and test for 1.4.0

Thanks for the review and comments.

[eum2o]

some promotion

I'll present your feature/repo at my workplace next week. We have most of our code implemented in OSGi plug-ins simply because we mostly develop inhouse desktop applications using Eclipse RCP. We now have use cases where we need to reuse that code outside the OSGi ecosystem (plain Java for now - maybe .NET in the future). So what you're doing here is exactly what we currently need (IMO - I hope the colleagues will agree). Until now, we're not working with remote services, Karaf, gRPC or BndTools, yet. If we starting using this remote service/gRPC toolchain, we will probably at least contribute bug issues and improvement suggestions - and whatever else that makes sense.

bndtools tooling (with automatic execution of protoc on every .proto source change) [...] protofile Eclipse editor plugin

I stumbled upon those things when researching about this topic. I haven't had time to check them out, yet. But definitively will do so!

be better to have one project for the api

Yes, that completely makes sense and I get your point. The examples are just fine and very helpful in their current state.

with the bndtools code generation [...]

This sounds really helpful when working with .proto files and protoc. I will give this a try at the latest when we actually starting using gRPC. Thanks for taking your time for this and all the information you provide (also in your blog/youtube - which I have just noticed). I will keep a close eye on what's happening in this area.

[scottslewis]

Hi Martin,

some promotion

I'll present your feature/repo at my workplace next week. We have most of our code implemented in OSGi plug-ins simply because we mostly develop inhouse desktop applications using Eclipse RCP. We now have use cases where we need to reuse that code outside the OSGi ecosystem (plain Java for now - maybe .NET in the future). So what you're doing here is exactly what we currently need (IMO - I hope the colleagues will agree). Until now, we're not working with remote services, Karaf, gRPC or BndTools, yet. If we starting using this remote service/gRPC toolchain, we will probably at least contribute bug issues and improvement suggestions - and whatever else that makes sense.

bndtools tooling (with automatic execution of protoc on every .proto source change) [...] protofile Eclipse editor plugin

I stumbled upon those things when researching about this topic. I haven't had time to check them out, yet. But definitively will do so!

I'm still working on videos and docs about putting this all together for bndtools code generation. Here are some of the resources that exist now...but I'm working on additional documentation. If you are willing/able to contribute (as you have already wrt examples, etc) I'll welcome anything. Just let's communicate about what is going on so everything gets covered:

1. Creating a workspace using the ECF remote services workspace template: https://wiki.eclipse.org/Bndtools_Support_for_Remote_Services_Development . The github repo referenced is here: https://github.com/ECF/bndtools.workspace and there is now a short video referenced on the page here: https://www.youtube.com/watch?v=rqWTHB17Pc4

2. Using the grpc project templates (now updated for 1.4.0) to generate api (from protofile in template); create osgi impl and consumer projects...again from project templates included in the ECF remote services workspace.

Here's a video that show this...which I may change, shorten and/or amend: https://www.youtube.com/watch?v=4-f4xQBlKr0

There's also a doc version of what's covered in this video, but I can't find right now.

be better to have one project for the api

Yes, that completely makes sense and I get your point. The examples are just fine and very helpful in their current state.

with the bndtools code generation [...]

This sounds really helpful when working with .proto files and protoc. I will give this a try at the latest when we actually starting using gRPC. Thanks for taking your time for this and all the information you provide (also in your blog/youtube - which I have just noticed). I will keep a close eye on what's happening in this area.

Thanks. I will try to be responsive as you and company discover bugs and/or have additional needs. The tooling aspect of this is just beginning to come together with bndtools. Note that I've already tested including a plain 'ol java project in a bndtools workspace...and adding project-level dependency to the api project (where java classes are generated automatically from protofile). That all works great fwiw.

I know it's often a struggle to get folks to use new tooling (e.g. bndtools)....and there are certainly still wrinkles but I can tell you that the bndtools folks are very active at improving things/responding to problems these days...as I was one of the folks nagging them about adding the automatic code generation and now it's there.