OEX , ZPM and GitHub: Control, Quality, Community

Discussion

Stefan Cronje · Feb 10, 2023

Hi Community,

Please read this when you have the time and keep an open mind. Also, you are welcome to tell me it is a terrible idea.

The main reason for this post

If you look at NPM, it has 1.3 million packages. Now ask yourself, is this an achievement or a nightmare?
How many of those packages are no longer supported?
How many of those packages break compatibility on version changes?
How do you know which one to use?

My opinion on the purpose of Opensource

Opensource is there to enable software professionals with different skills sets, experience and background to contribute towards a common good, which is simple, reusable, maintainable, quality software packages.

Open Exchange

I have had a good look at OEX, and I have noticed a few things:

There are many packages.
There are duplications of functionality the packages provide.
Most packages only have one contributor.
Many packages haven't been updated in a long time.

Me on a tangent

Skip this part if you don't like it when people go on a tangent.

I think we are heading the NPM route. Quantity does not mean better. Quality does. We are all strong developers and individuals, and we all want to make a difference and contribute. We all also want that contest prize, to be honest. The contest prize I think pushes the people, which is good, but maybe contributes towards the single contributor packages. I am not saying the contests should stop, but maybe when doing something for a contest, try and do it and take part without making a new package. Chat with the Repo owner - I am sure the owner will not mind if you improve on their repo and get the prize.

Tangent done.

The proposal

More cohesive, simple, quality packages

Purge dead repos - Give the owner a chance to indicate if it is still relevant.
Consolidate similar packages into a better package
Partner up more

I want to provide a few examples, and I am not accusing anyone of anything, so please do not take it personally.

Triple slash - Great idea. TestCoverage - Great tool. Why are they not in a single "powerful" package?
OpenAPI Suite - Great. There are many other swagger and API tools. Why do we not merge them all into a single great package for REST

Dependencies - it seems like contributors are afraid to create a dependency on other packages - I can understand this, but duplicating something is not always the best idea. Communicate with that repo owner and ensure the dependency is not broken by new changes.

People's strengths. I will use myse4lf as an example. I can write some good code generators and other ObjectScript things. I am not a UI developer, I do not do UI for reasons beyond my control. But for some packages I might have wanted to have an UI. Other people are very good again at UI, but maybe less good on ObjectScript. Let's throw out a posting asking for someone to help on a repo - maybe a new type of posting, like "Resrouce Request".

That is it from me, let's discuss.

Elena E · Feb 10, 2023

@Stefan Cronje ,

thank you for raising such a great and relevant point. I highly appreciate your ideas and the fact that you are not indifferent. The points you raised are very important and I'd like to share my thoughts on them.

Regarding many packages not being updated in a long time and likely not working, we have recently implemented a process of checking app workability, inspired by @Robert Cemper idea. This process was just released yesterday, but we haven't announced it yet (coming soon). With this process in place, we can constantly monitor the apps and remove any that are not working from the list until they are fixed.

The issue of most packages only having one contributor is a significant and deep problem that requires consideration. We need to figure out how to bring people together to collaborate on app creation and encourage teamwork. While it's easier to complete small tasks individually, great and sustainable projects always require a team. We will think about how we can provide a platform that facilitates collaboration among developers and potentially UI designers and other roles that may be useful.

As for duplications of functionality in the packages, this is also a valid concern. We will introduce a check for repeatability in functionality and inform developers of any duplications. We may even bring developers together if they have similar ideas.

With over 700 packages available, now is the right time to implement these changes, as we can consider this a critical mass.

Once again, thank you for your ideas. It would be great to have a brainstorming meeting to discuss these issues with anyone who is interested in the near future.

3 0

Robert Cemper · Feb 10, 2023

Hi @Stefan Cronje !
I share many of the concerns you raised:

aged and outdated and dead packages
extra points are given for PR and Merges, but most PS are ignored
besides my own 402 reviews only 80 others
no options for a release-related review
enough dead links
missing an ISC-controlled repo

2 0

Thank you for the information and the proposal to have a brainstorming session as a community.

0 0

Lorenzo Scalese · Feb 10, 2023

Thank you for this very interesting post @Stefan Cronje

You're right, for example, there are packages today useless because IRIS now provides features.

I have a package which I think should be deleted (to discuss with admin).

About OpenAPI-Suite, I use a lot of dependencies with packages owned by myself or not to avoid code duplication (for this development I make a pull request to an existing package)

A first clean, could be not shown in OpenExchange these dependencies: openapi-client-gen, openapi-server-gen, openapi-common-lib, swagger-validator-cli, swagger-converter-cli. It's useless, peoples need the complete solution, not the dependencies.

If we create a larger community package on the "REST" topic, of course, I will contribute to the integration.

I understand completely. As a side comment, which is a bit off topic, I like what you've done and think I will contribute to it when time allows. I created the old Dynamic Object Adapter pacakge for Ensemble. There are things in there we can add to the OpenAPI suite if needed, for example the reverse of API first. Existing class definitions to swagger type thing.

1 0

Timothy Leavitt · Feb 10, 2023

@Stefan Cronje re: existing class definitions to Swagger, that's covered in https://github.com/intersystems/isc-rest

I agree on most of this. A few comments:

"Triple slash - Great idea. TestCoverage - Great tool. Why are they not in a single "powerful" package?"

These tools solve different, distinct problems (although both are related to unit testing). Tools that solve different, distinct problems should be separate packages. If you want to write a single "powerful" package that depends on both of them and maybe adds some glue between them, feel free!

Dependencies:

Packages should use semantic versioning, and dependencies in IPM can be declared in a way that adheres to the semantic versioning contract. It's the responsibility of the dependency repo owner to follow this, and if you don't trust them you can just lock down to the version you tested with.

Also, I'm hoping to get around to reviewing your TestCoverage PR soon - just trying to deal with some CI infrastructure issues, and my day job keeps interfering.

Thank you for your response on this. I see what you are saying regarding the different problems being solved.

This is basically the agenda I am pushing - let's talk about packages and what should be and should not be packaged together, what we need, etc. BUT without adding "red tape" that will demotivate community members from contributing.

I seem to be giving a lot of 2c today.

Alex Woodhead · Feb 10, 2023

Could we have Community Projects that incubate, for example in the way that the Apache Software Foundation incubates sub-projects.
A sub-project acts as structure to decide to Adopt requirements flagged on the "Ideas Portal".
They act as a focus of activity that people can join and leave over time.
They provide a roadmap of what is intended to deliver
They would define a membership of both 1) Developers 2) Quality Assurance

One benefit could be that feedback to a Project might feel less personal, and more about helping individuals enjoy collaborating and networking.

Yuri Marx · Feb 10, 2023

Great idea!!!

Evgeny Shvarov · Feb 10, 2023

Hi @Stefan Cronje !

Thanks for raising this topic! In fact we are aware of the problem and introduce different features and services that can increase the level of quality of the applications.

1. We give the bonus for the ObjectScript quality, and all the templates contain ObjectScript linter, that free of charge monitors ObjectScript public code upon a set of rules.

And we introduced a special filter on OEX that shows only such apps.

2. We encourage developers to review applications. And to simplify the review, we encourage usage IPM (and we have a special filter) and applications with online demos, as this saves the time of reviewers greatly.

3. We'd love to see projects with multiple contributions and the first thing that could help contributors is to easily setup the dev environment - and here nothing is best than docker environment. That's why we encourage docker usage so much and dockerfile is presented in each and every template.

4. We have InterSystems supported samples and templates to show the set of trusted applications and establish the code guidelines.

Actually, you can see the focus of the enhancements if you look what bonuses we suggest for the applications in contests - they are all connected to what we think can increase the quality of applications and engage its usage.

Also, we encourage participants of the contests to apply again with the existing apps and add new features into them. And developers do that.

On the team forming: we invite developers to participate as teams - and some of them do that. We have a team of 3 developers from Brazil who participate in a recent contest with iris-trippleslash. But of course it'd be great to see more teams like that.

Evgeny Shvarov · Feb 11, 2023

Regarding it. First of all, thank you for the feedback. This is very important and we are looking for such a feedback all the time.

Yes, we have 700 apps, and 4 years ago we had zero and contests helped to have some variety.

Also, I hope every member of this developer community could participate in a contest with the new app, as it as we think gives a habit to stand-out from your own dev-bubble and consider docker, ipm, online demos, unit-testing and sometimes provide a feedback!

But on contrary we encourage very much re-applying with already created apps and we see several improved apps even in the current contest.