| commit | 9347fe5df6ac0e579df7b75421578eedb5342545 | [log] [tgz] |
|---|---|---|
| author | cdmikechen <cdmikechen@apache.org> | Sun Dec 11 13:56:03 2022 +0800 |
| committer | Kevin <pingsutw@apache.org> | Mon Dec 26 05:16:10 2022 +0000 |
| tree | f8599347c482026862c289ab850fcdffd80e90d7 | |
| parent | 4d234b01e7383a22b5e0ded0d58966e13c8eb16a [diff] |
SUBMARINE-1138. New SSO function based on OIDC ### What is this PR for? Use pac4j to support OIDC and default login action, and fix some user rest api question. Currently, it is a preview version, which is mainly modified for the background and adapted to the front-end processing. The current purpose is to summarize the core of the modification and test cicd. So please do not merge the current changes! ### What type of PR is it? Improvement ### Todos * [x] - User `pac4j-oidc` to support OIDC SSO based on cookie/session * [x] - Support rest api with header token * [x] - Front end modification. The 302 redirection of httpclient is not handled at present * [x] - Remove jdk1.8 support * [x] - Optimized cookie configuration * [x] - automatically create new user when logged in * [x] - Support clustering session by jdbc * [x] - Change mybatis log to SLF4J * [x] - Add some tests * [x] - Add some more documents about oidc support ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-1138 ### How should this be tested? Need to add some test later. ### Screenshots (if appropriate) ### Questions: * Do the license files need updating? No * Are there breaking changes for older versions? Yes * Does this need new documentation? Yes Author: cdmikechen <cdmikechen@apache.org> Signed-off-by: Kevin <pingsutw@apache.org> Closes #1019 from cdmikechen/SUBMARINE-1138-0.8.0-pacj4j5.7 and squashes the following commits: 359109b8 [cdmikechen] SysUserService singleton 3501790e [cdmikechen] Add cookie document 0a455761 [cdmikechen] Fix document ad9d1902 [cdmikechen] Add test d1a3304e [cdmikechen] revert authType b5752418 [cdmikechen] remove SUBMARINE_AUTH_TYPE in image c5526736 [cdmikechen] Fix test error 2e296587 [cdmikechen] Remove derby and upgrade jdk11 version c8644cea [cdmikechen] update jdk11 2803bda4 [cdmikechen] Adjustment code e9a1b8ac [cdmikechen] Support jdk11 and pac4j 5.6.1 Add cookie samesite/httponly/securite eef13732 [cdmikechen] Test python-sdk 2ce98c1c [cdmikechen] Dealing with automatic user creation 1c98d2fc [cdmikechen] Commit for python check fix 9ecb7cbc [cdmikechen] Add api paths auth checks 220c49a0 [cdmikechen] Change auth type to flow type 3df16b4b [cdmikechen] Use servlet to replace static auth type check js 94099147 [cdmikechen] Handle front-end workbench oidc support 8b786954 [cdmikechen] deal with 401 16fe1a17 [cdmikechen] Add @Context to fix error 90eb5c5c [cdmikechen] Add token to rest api header 0f8f2636 [cdmikechen] Add oidc backend support(excluding the addition of oidc users)
Apache Submarine (Submarine for short) is an End-to-End Machine Learning Platform to allow data scientists to create end-to-end machine learning workflows. On Submarine, data scientists can finish each stage in the ML model lifecycle, including data exploration, data pipeline creation, model training, serving, and monitoring.
Some open-source and commercial projects are trying to build an end-to-end ML platform. What's the vision of Submarine?
Theodore Levitt once said:
“People don’t want to buy a quarter-inch drill. They want a quarter-inch hole.”
experiment on prem or cloud via easy-to-use UI/API/SDK.experiment and dependencies of environment.As mentioned above, Submarine attempts to provide Data-Scientist-friendly UI to make data scientists have a good user experience. Here're some examples.
# New a submarine client of the submarine server submarine_client = submarine.ExperimentClient(host='http://localhost:8080') # The experiment's environment, could be Docker image or Conda environment based environment = EnvironmentSpec(image='apache/submarine:tf-dist-mnist-test-1.0') # Specify the experiment's name, framework it's using, namespace it will run in, # the entry point. It can also accept environment variables. etc. # For PyTorch job, the framework should be 'Pytorch'. experiment_meta = ExperimentMeta(name='mnist-dist', namespace='default', framework='Tensorflow', cmd='python /var/tf_dist_mnist/dist_mnist.py --train_steps=100') # 1 PS task of 2 cpu, 1GB ps_spec = ExperimentTaskSpec(resources='cpu=2,memory=1024M', replicas=1) # 1 Worker task worker_spec = ExperimentTaskSpec(resources='cpu=2,memory=1024M', replicas=1) # Wrap up the meta, environment and task specs into an experiment. # For PyTorch job, the specs would be "Master" and "Worker". experiment_spec = ExperimentSpec(meta=experiment_meta, environment=environment, spec={'Ps':ps_spec, 'Worker': worker_spec}) # Submit the experiment to submarine server experiment = submarine_client.create_experiment(experiment_spec=experiment_spec) # Get the experiment ID id = experiment['experimentId']
submarine_client.get_experiment(id)
submarine_client.wait_for_finish(id)
submarine_client.get_log(id)
submarine_client.list_experiments(status='running')
For a quick-start, see Submarine On K8s
(Available on 0.5.0, see Roadmap)
If you want to know more about Submarine's architecture, components, requirements and design doc, they can be found on Architecture-and-requirement
Detailed design documentation, implementation notes can be found at: Implementation notes
Read the Apache Submarine Community Guide
How to contribute Contributing Guide
Login Submarine slack channel: https://join.slack.com/t/asf-submarine/shared_invite
Issue Tracking: https://issues.apache.org/jira/projects/SUBMARINE
What to know more about what's coming for Submarine? Please check the roadmap out: https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap
From here, you can know the changelog and the issue tracker of different version of Apache Submarine.
Apache submarine: a unified machine learning platform made simple at EuroMLSys '22
The Apache Submarine project is licensed under the Apache 2.0 License. See the LICENSE file for details.