.. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at .. http://www.apache.org/licenses/LICENSE-2.0 .. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. .. Kyuubi documentation master file, created by sphinx-quickstart on Wed Oct 28 14:23:28 2020. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. HOME ==== Kyuubi™ is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of `Apache Spark™ `_. .. raw:: html
In general, the complete ecosystem of Kyuubi falls into the hierarchies shown in the above figure, with each layer loosely coupled to the other. For example, you can use Kyuubi, Spark and `Apache Iceberg `_ to build and manage Data Lake with pure SQL for both data processing e.g. ETL, and analytics e.g. BI. All workloads can be done on one platform, using one copy of data, with one SQL interface. Kyuubi provides the following features: Multi-tenancy ------------- Kyuubi supports the end-to-end multi-tenancy, and this is why we want to create this project despite that the Spark `Thrift JDBC/ODBC server `_ already exists. 1. Supports multi-client concurrency and authentication 2. Supports one Spark application per account(SPA). 3. Supports QUEUE/NAMESPACE Access Control Lists (ACL) 4. Supports metadata & data Access Control Lists Users who have valid accounts could use all kinds of client tools, e.g. Hive Beeline, `HUE `_, `DBeaver `_, `SQuirreL SQL Client `_, etc, to operate with Kyuubi server concurrently. The SPA policy makes sure 1) a user account can only get computing resource with managed ACLs, e.g. `Queue Access Control Lists `_, from cluster managers, e.g. `Apache Hadoop YARN `_, `Kubernetes (K8s) `_ to create the Spark application; 2) a user account can only access data and metadata from a storage system, e.g. `Apache Hadoop HDFS `_, with permissions. Ease of Use ------------ You only need to be familiar with Structured Query Language (SQL) and Java Database Connectivity (JDBC) to handle massive data. It helps you focus on the design and implementation of your business system. Run Anywhere ------------ Kyuubi can submit Spark applications to all supported cluster managers, including YARN, Mesos, Kubernetes, Standalone, and local. The SPA policy also make it possible for you to launch different applications against different cluster managers. High Performance ---------------- Kyuubi is built on the Apache Spark, a lightning-fast unified analytics engine. - **Concurrent execution**: multiple Spark applications work together - **Quick response**: long-running Spark applications without startup cost - **Optimal execution plan**: fully supports Spark SQL Catalyst Optimizer, Authentication & Authorization ------------------------------ With strong authentication and fine-grained column/row level authorization, Kyuubi keeps your system and data secure. High Availability ----------------- Kyuubi provides both high availability and load balancing solutions based on Zookeeper. .. toctree:: :caption: Admin Guide :maxdepth: 2 :glob: quick_start/index deployment/index Security monitor/index tools/index .. toctree:: :caption: User Guide :maxdepth: 2 :glob: Clients & APIs SQL References .. toctree:: :caption: Extension Guide :maxdepth: 2 :glob: Extensions .. toctree:: :caption: Connectors :maxdepth: 2 :glob: connector/index .. toctree:: :caption: Kyuubi Insider :maxdepth: 2 overview/index .. toctree:: :caption: Contributing :maxdepth: 2 develop_tools/index community/index .. toctree:: :caption: Appendix :maxdepth: 2 appendix/index