Help Center > > User Guide> Cluster Management> Creating a Hybrid Cluster

Creating a Hybrid Cluster

Updated at: Nov 05, 2019 GMT+08:00

On the CCE console, you can easily create Kubernetes clusters. Kubernetes can manage container clusters at scale. A cluster manages a group of node resources.

Support for hybrid clusters is available now. Hybrid clusters can manage VM nodes, bare-metal nodes (also called PM nodes), and both. The previous VM Cluster card on the Clusters page will be renamed as Hybrid Cluster.

PM nodes can be created only after the hybrid cluster is created and meets the following conditions:

  • The cluster version is v1.13.7 or later if the networking model is tunnel network.
  • The cluster version is v1.11.7 or later if the networking model is VPC network.
  • All IP addresses are not IPv6.
  • PM nodes in the cluster are billed on a yearly/monthly basis.

Preparation

  • Before creating your first cluster, you must create a VPC. If you already have an available VPC, skip this preparatory step.

    A VPC provides an isolated, configurable, and manageable virtual network for CCE clusters. For details, see Creating a VPC.

  • Create a key pair, which will be used for identity authentication upon remote node login.

    If you use a password to log in to a node, skip this step. For details, see https://support.huaweicloud.com/en-us/usermanual-ecs/en-us_topic_0014250631.html.

  • Plan the container CIDR block and service CIDR block before creating a cluster. The CIDR block is a one-time configuration and cannot be changed after the cluster is created. If you want to use another container CIDR block, you have to create a new cluster and assign the new container CIDR block to the cluster.

Precautions for Creating a Cluster

Some basic resources are created during cluster creation, as shown in the following table.

Table 1 Precautions for creating a cluster

Resource

Description

Masters and related resources

Associated with CCE resource tenants, and invisible to you.

ECSs (optional)

An ECS corresponds to a cluster node that provides computing resources.

An ECS is named in the format of Cluster name-Random number. The name format is user-defined. ECSs created in batches are named in the format of Cluster name-Random number 1-Random number 2.

Security groups

Two security groups are created for a cluster: one for managing master nodes, and the other for managing worker nodes.

NOTICE:

Do not delete the security group settings and security group rules configured during cluster creation. Otherwise, the cluster will exhibit unexpected behavior.

  1. Security group for master nodes

    Name format: Cluster name-cce-control-Random number

    Functions:

    • Allows outbound traffic.
    • Allows other nodes to access Kubernetes services of masters.
  2. Security group for nodes

    Name format: Cluster name-cce-node-Random number

    Functions:

    • Allows outbound traffic.
    • Allows remote login to Linux or Windows OS using ports 22 and 3389.
    • Allows communication between Kubernetes components using ports 4789 and 10250.
    • Allows Kubernetes to provide services for external systems using ports 30000 and 32767.
    • Allows communication between nodes in the same security group.

Disks (optional)

Two disks are configured for each node. One is the system disk, and the other is the data disk used to run Docker.

Elastic IP address (optional)

Any node that requires access to external networks must have an elastic IP address (EIP).

Procedure

  1. Log in to the CCE console. Choose Dashboard > Buy Cluster to open the Buy Hybrid Cluster page. An alternative way to open that page is to choose Resource Management > Clusters in the navigation pane and click Buy under Hybrid Cluster.

    Figure 1 Cluster Management > Hybrid Cluster > Buy

  2. Set the parameters listed in Table 2. The parameters marked with an asterisk (*) are mandatory.

    Table 2 Parameters for creating a cluster

    Parameter

    Description

    *Billing Mode

    • Yearly/Monthly: Resources will be billed on a yearly or monthly basis. Yearly/monthly billed clusters cannot be deleted after creation. To stop using these clusters, go to the Billing Center and unsubscribe them.
    • Pay-per-use: Resources will be billed on an hourly basis. Pay-per-use is selected by way of example.

    *Region

    To minimize network latency and resource access time, select the nearest region. Cloud resources are region-specific and cannot be used across regions through internal network connections.

    *Enterprise project

    This parameter is displayed only for enterprise users who have enabled Enterprise Project Management. Ensure that the selected enterprise project contains resources required for cluster creation, such as VPC.

    For more information, see Enterprise Management.

    *Cluster Name

    Name of the new cluster, which cannot be changed after the cluster is created.

    A cluster name contains 4 to 128 characters starting with a letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed.

    *Version

    Kubernetes community baseline version. The latest version is recommended. For details about version changelog, see Changelog for Upgrading Clusters.

    *Management Scale

    Maximum number of nodes that can be managed by a cluster.

    If you select 50 nodes, the cluster can manage a maximum of 50 nodes. The cluster management scale cannot be modified after a cluster is created. Exercise caution when creating a cluster.

    Each cluster contains at least one master node and at least one worker node. A worker node is a cloud server.
    • Master node: controls worker nodes in the cluster. The master node is automatically created along with the cluster, and manages and schedules the entire cluster.
    • Worker node: a node that runs the workload you deploy. You can createbuy a new worker node. The master node assigns a worker node to each deployable component of your workload. When a worker node is down, the master node migrates the workload to another worker node.

    *High Availability

    • Yes: An HA cluster will be created. Typically, an HA cluster has three master nodes. The cluster is still available even when a master node is down.
    • No: The cluster has only one master node. If the master node is down, the cluster stops serving new workloads, but existing workloads are not affected.
    The HA setting is a one-time configuration and cannot be changed after the cluster is created. If you want to use another HA setting for the cluster, you have to create a new cluster that has exactly the same parameter settings (except for the High Availability parameter) as the current cluster. Set this parameter based on the site requirements.
    • In production environments, it is advised to set High Availability to Yes to improve cluster's disaster recovery capabilities.
    • In R&D and testing environments that do not require high reliability, it is not always necessary to set High Availability to Yes.

    *VPC

    VPC where the new cluster is located. A VPC provides a secure and logically isolated network environment.

    If no VPC is available, click Create VPC to create a VPC. After the VPC is created, click the refresh icon.

    *Subnet

    The subnet where cluster nodes will run. A subnet improves network security by providing exclusive network resources that are isolated from other networks. For details about the relationship between VPCs, subnets, and clusters, see Cluster Overview.

    The selected subnet cannot be modified after the cluster is created. Exercise caution when selecting a subnet.

    *Network Model

    • Tunnel network: The container network is an overlay tunnel network on top of a VPC network and uses the VXLAN technology. This network model is applicable when there is no high requirements on performance. VXLAN encapsulates Ethernet packets as UDP packets for tunnel transmission. Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications.
    • VPC network: The container network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. Each node is assigned a CIDR block of a fixed size. VPC networks are free from tunnel encapsulation overhead and outperform container tunnel network models. In addition, as VPC routing includes routes to node IP addresses and container network segment, container instances in the cluster can be directly accessed from outside the cluster.

    *Container Network Segment

    An IP address range that can be allocated to container instances.

    • If Automatically select is deselected, you must select a CIDR block. If the CIDR block you select conflicts with a subnet CIDR block, the system prompts you to select another CIDR block. The recommended CIDR blocks are 10.0.0.0/12-19, 172.16.0.0/16-19, and 192.168.0.0/16-19.

      If different clusters share a container CIDR block, an IP address conflict will occur and access to applications will fail.

    • If Automatically select is selected, the system automatically assigns a CIDR block that does not conflict with any subnet CIDR block.

    The mask of the container CIDR block must be appropriate. It determines the number of available nodes in a cluster. A too small mask value will cause the cluster to soon fall short of nodes. After the mask is set, the estimated maximum number of containers supported by the current CIDR block will be displayed.

    Service Network Segment

    CIDR block of Kubernetes services.

    • Default: The default CIDR block 10.247.0.0/16 will be used.
    • Custom: Manually set a network segment and mask based on service requirements. The mask determines the maximum number of service IP addresses available in the cluster.

    Authorization Mode

    By default, RBAC is selected. Read the CCE Role Management and select I an aware of the above limitations and read the CCE Role Management Instructions.

    After RBAC is enabled, IAM users access resources in the cluster according to fine-grained permissions policies.

    Authentication Mode

    The authentication mechanism controls user permission on resources in a cluster. For example, you can grant user A the read and write permissions on applications in a specified namespace, while granting user B the read permission on resources in a cluster. For details about role-based permission control, see 3.7- Cluster Management Permission Control.

    • By default, X.509 authentication instead of enhanced authentication is enabled. X.509 is a standard defining the format of public key certificates. X.509 certificates are used in many Internet protocols.
    • If permission control on a cluster is required, select Enhanced authentication and then Authenticating Proxy.

      Click Upload next to CA Root Certificate to upload a valid certificate. Select the check box to confirm that the uploaded certificate is valid.

      If the certificate is invalid, the cluster cannot be created. The uploaded certificate file must be smaller than 1 MB and in .crt or .cer format.

    Cluster Description

    Optional. Enter the description of the new container cluster.

    Advanced Settings

    Click to show advanced functions.

    Advanced settings will be displayed only if they are supported in the selected AZ. For example, if the flavor of cluster's master node does not support the multi-AZ feature, Multiple AZs will not be displayed.

    The following parameters are supported:

    ipv6

    After this option is enabled, an IPv6 cluster will be created.

    Multiple AZs

    The multi-AZ deployment of master nodes achieves disaster tolerance on the cluster management plane at a certain sacrifice of cluster performance.

    • : disabled. Master nodes are deployed in the same AZ. If the AZ is down, the cluster stops serving new workloads, but existing workloads are not affected.
    • : enabled. Master nodes are distributed across multiple AZs. When one of the AZs is down, the cluster can continue to serve new workloads.

    Service Forwarding Mode

    • iptables: Traditional kube-proxy uses iptables rules to implement service load balancing. In this mode, too many iptables rules will be generated when many services are deployed. In addition, non-incremental updates will cause a latency and even obvious performance issues in the case of heavy service traffic.
    • ipvs: kube-proxy mode optimized by Huawei to achieve higher throughput and faster speed. This mode supports incremental updates and can keep connections uninterrupted during service updates. It is suitable for large-sized clusters.
    NOTE:
    • ipvs provides better scalability and performance for large clusters.
    • Compared with iptables, ipvs supports more complex load balancing algorithms such as least load first (LLF) and weighted least connections (WLC).
    • ipvs supports server health checking and connection retries.

    Maximum Pods

    This parameter is displayed only for clusters whose Network Model is set to VPC network.

    This parameter indicates the maximum number of pods that can be created on a cluster node. This is a one-time configuration and cannot be modified later. The default value is 256. After the cluster is created, the value of Max Pods in the Advanced Settings area of the Create Node page must be lesser than or equal to the value specified here.

    Resource Tags

    You can classify resources by their tags.

    You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. For details, see Creating Predefined Tags.

    CPU Management Policy

    • On: Exclusive CPU cores can be allocated to workload pods. Select On if your workload is sensitive to latency in CPU cache and scheduling.
    • Off: Exclusive CPU cores will not be allocated to workload pods. Select Off if you want a large pool of shareable CPU cores.

    *Validity Period

    For a yearly/monthly billed cluster, set the required duration.

  3. Click Next. On the Node page, set the following parameters.

    • Node
      • Create now: A specified number of nodes will be created along with the cluster. The Nodes parameter indicates the quantity of nodes.

      • Create later: Create an empty cluster without creating nodes. Click Next.
    • Billing Mode: Select Yearly/Monthly or Pay-per-use.

      Nodes created along with the cluster must inherit the billing mode from the cluster. For example, if the billing mode of the cluster is pay-per-use, then nodes created along with the cluster must be billed on the pay-per-use basis. For details, see Buying a Pay-Per-Use Node and Buying a Yearly/Monthly Billed Node.

      Yearly/monthly billed nodes cannot be deleted after creation. To stop using these nodes, go to the Billing Center and unsubscribe them.

    • Current Region: geographic location of the nodes to be created.
    • AZ: Set this parameter based on the site requirements. An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network.

      To enhance workload availability, create nodes in different AZs.

    • Node Type
      • VM node: A VM node will be created in the cluster.
      • Bare-metal node: Bare-metal nodes cannot be created along with a cluster. You can add bare metal nodes to a cluster only after the cluster is created. For more information, see Bare Metal Server or Creating a BMS Cluster.
        Bare-metal nodes can be created only after the hybrid cluster is created and meets the following conditions:
        • All IP addresses are not IPv6.
        • The cluster version is v1.11.7 or later (network model is VPC network), or v1.13.10 or later (network model is tunnel network).
        • Bare-metal nodes in the cluster are billed on a yearly/monthly basis.

        For details on how to buy bare-metal nodes, see Buying a Yearly/Monthly Billed Node.

    • Node Name: Enter a node name. If you change the node name on the ECS console after the node is created, be sure to synchronize the new node name from ECS to CCE. For details, see Synchronizing Node Data.
    • Specifications: Select node specifications that best fit your business needs.
      • General-purpose: provides general computing, storage, and network configurations that can meet a majority of scenarios. General-purpose nodes can be used for web servers, workload development, workload testing, and small databases.
      • Memory-optimized: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive.
      • GPU-accelerated: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. Currently, only clusters of v1.11 support GPU-accelerated nodes. If the cluster version is v1.13 or later, GPU-accelerated is not displayed on the page.
      • High-performance computing: provides stable and ultra-high computing performance and is suitable for scientific computing and workloads that demand ultra-high computing and throughput.
      • General computing-plus: provides stable performance and exclusive resources to enterprise-level workloads with high and stable computing performance.
      • Disk-intensive: supports local disk storage and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads.
      • Ultra-high I/O: provides ultra-low SSD access latency and ultra-high IOPS performance. This type of specifications is suitable for high-performance relational databases, NoSQL databases (such as Cassandra and MongoDB), and Elasticsearch.
      Figure 2 Selecting node specifications

      To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see Formula for Calculating the Reserved Resources of a Node.

    • OS: Select the operating system (OS) of the nodes to be created.

      Reinstalling OS or modifying OS configurations could make nodes unavailable. Exercise caution when performing these operations. For more information, see Risky Operations on Cluster Nodes.

    • VPC: This parameter is displayed only for clusters of v1.13.10-r0 and later.
    • Subnet: A subnet improves network security by providing exclusive network resources that are isolated from other networks. You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets.

      This parameter is displayed only for clusters of v1.13.10-r0 and later.

    • EIP: An independent public IP address. If the nodes to be created require Internet access, select Automatically assign or Use existing.
      • Do not use: A node without an EIP cannot access the Internet. It can be used only as a cloud server for deploying services or clusters on a private network.
      • Automatically assign: An EIP with specified configurations is automatically assigned to each node. If the number of EIPs is less than the number of nodes, the EIPs are randomly bound to the nodes.

        Set the specifications, required quantity, billing mode, and bandwidth as required. When creating an ECS, ensure that the elastic IP address quota is sufficient.

      • Use existing: Existing EIPs are assigned to the nodes to be created.
    • System Disk and Data Disk: Set the disk space of the nodes.
      • The system disk capacity ranges from 40 GB to 1,024 GB. The default value is 40 GB.
      • Data disk:
        • The EVS disk capacity ranges from 100 GB to 32,678 GB. The default value is 100 GB.

          If you select Disk Space Allocation, you can allocate data disk space for storing Docker and Kubelet resources.

        • Local disk: This parameter is displayed only for disk-intensive nodes in clusters of Kubernetes v1.13.10-r0 or later. Local disks may break down and does not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks. The local disk parameters are as follows:
          • Disk Mode: If the node type is disk-intensive, the supported disk mode is HDD. If the node type is ultra-high I/O, the supported disk mode is SSD.
          • Read/Write Mode: The serial and parallel modes are supported. If there are multiple local disks, you can set the read/write mode. Serial indicates that data is read and written in linear mode. When a disk is used up, the next disk is used. Parallel indicates that data is read and written in striping mode, allowing multiple local disks to be read and written at the same time.
          • Kubernetes Space: the percentage of data disk space allocated for storing Docker and Kubelet resources. Docker resources include Docker images and image metadata; Kubelet resources include pod configuration files, secrets, and EmptyDirs.
          • User Space: the percentage of the local disk space that is not allocated to Kubernetes.
        • The percentage of k8s space and user space must be equal to 100%. You can modify the percentage by clicking .
        • By default, disks run in the direct-lvm mode. If data disks are removed, the loop-lvm mode will be used and this will impair system stability. For more information, click here.

      System disks and data disks deliver three levels of I/O performance:
      • Common I/O: uses Serial Advanced Technology Attachment (SATA) drives to store data. EVS disks of this level provide reliable block storage and a maximum IOPS of 1,000 per disk. They are suitable for key applications.
      • High I/O: uses serial attached SCSI (SAS) drives to store data. EVS disks of this level provide a maximum IOPS of 3,000 and a minimum read/write latency of 1 ms. They are suitable for Relational Database Service (RDS), NoSQL, data warehouse, and file system applications.
      • Ultra-high I/O: uses solid state disk (SSD) drives to store data. EVS disks of this level provide a maximum IOPS of 20,000 and a minimum read/write latency of 1 ms. They are suitable for RDS, NoSQL, and data warehouse applications.
    • Login Mode: Currently, you can use a password or key pair to log in to the node.
      • If the login mode is Password: The default username is root. Enter the password for logging to the node and confirm the password.

        Be sure to remember the password as you will need it when you log in to the node.

      • If the login mode is Key pair, select a key pair for logging to the node.

        A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click Create a key pair. For details on how to create a key pair, see Creating a Key Pair.

        Figure 3 Key pair
    • Advanced ECS Settings (optional): Click to show advanced ECS settings.
      • ECS Group: Select an existing ECS group, or click Create ECS Group to create a new one. After the ECS group is created, click the refresh icon.

        An ECS group allows you to create ECSs on different hosts, thereby improving service reliability.

      • Resource Tags: By adding tags to resources, you can classify resources.

        You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. For details, see Creating Predefined Tags.

        CCE will automatically create the "CCE-Dynamic-Provisioning-Node=node id" tag. A maximum of 5 tags can be added.

      • Agency: An agency is created by a tenant administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. . To authorize ECS or BMS to call cloud services, select Cloud service as the agency type, click Select, and then select ECS BMS.
      • Pre-installation Script: Enter a maximum of 1,000 characters.

        The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may not be installed successfully. The script is usually used to format data disks.

      • Post-installation Script: Enter a maximum of 1,000 characters.

        The script will be executed after Kubernetes software is installed and will not affect the installation. The script is usually used to modify Docker parameters.

      • Add Data Disk: Click Add Data Disk to add a data disk and set the capacity of the data disk. To simplify disk formatting, you can enter the disk formatting command in the input box of Pre-installation Script. For details, see How Do I Format a Data Disk Using Command Line Injection?.
      • Subnet IP Address: Select Automatically assign IP address (recommended) or Manually assigning IP addresses.
    • Advanced Kubernetes Settings: (Optional) Click to show advanced ECS settings.
      • Max Pods: The maximum number of pods that can be created on a node, including the system's default pods. Value range: 16 to 250.

        This maximum limit prevents the node from being overloaded by managing too many pods.

      • insecure-registries: Click Add insecure-registry and enter the address of the image repository.

        Add the address of the custom image repository with no valid SSL certificate to the docker startup option to avoid unsuccessful image pulling from the personal image repository. The address is in the format of IP address:Port number (or domain name). Post-installation script and insecure-registries cannot be used together.

      • Data Space Per Container: the maximum data space that can be used by a container. Value range: 10 GB to 80 GB. If the value of this field is larger than the data disk space allocated to Docker, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker. This parameter is displayed only for clusters of v1.13.10-r0 and later.
    • Nodes: The specified number of nodes cannot exceed the maximum number of nodes that can be managed by the cluster. To increase the quota limit on nodes, create a service ticket.
    • Validity Period: If the cluster billing mode is yearly/monthly, set the number of months or years for which you will use the new node.

  4. Click Next to install add-ons.

    System resource add-ons must be installed. Advanced functional add-ons are optional.

    You can also install optional add-ons after the cluster is created. To do so, choose Add-ons in the navigation pane of the CCE console and select the add-on you will install. For details, see 13 Add-on Management.

  5. Click Buy Now and confirm the specifications.

    Confirm the specifications you have configured.

  6. Confirm the specifications and price, and click Submit.

    It takes about 6 to 10 minutes to create a cluster. You can click Back to Cluster List to perform other operations on the cluster or click Go to Cluster Events to view the cluster details.

    • If the cluster wil be billed on a yearly/monthly basis, follow on-screen prompts to pay the order.

Related Operations

  • Create a namespace. You can create multiple namespaces in a cluster and organize resources in the cluster into different namespaces. These namespaces serve as logical groups and can be managed separately. For more information about how to create a namespace for a cluster, see Namespace.
  • Click the cluster name to view cluster details.
    Table 3 Cluster details

    Tab

    Description

    Cluster Details

    View the details and operating status of the cluster.

    Monitoring

    Check the CPU and memory usage of the cluster over the past 1 hour, 3 hours, or 12 hours.

    Events

    • View cluster events on the Events tab page.
    • Set search criteria. For example, you can set the time segment or enter an event name to view corresponding events.

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel