Starting from Source Code Locally

This document describes how to start qData from source code on your local machine. Please follow the steps below sequentially to complete environment preparation, service configuration, and startup.

---

1. Prerequisites

1.1 Install Docker and Docker Compose

Before installing qData, ensure your machine meets the following minimum requirements (it is recommended to install Docker-related components on a Linux server or virtual machine):

• CPU: >= 4 cores
• Memory (RAM): >= 8 GiB

⚠️ Note: Ensure Docker and Docker Compose are correctly installed and can be executed from the terminal.

2. Version Requirements

Component	Version Requirements
JDK	1.8
Maven	3.6+
yarn	v1.22.22+
Node.js	18.0+
Docker	19.03 or later
Docker Compose	1.28 or later

3. Clone the qData Repository

git clone https://gitee.com/qiantongtech/qData.git

4. Start Middleware Using Docker Compose

The qData backend depends on the following middleware:

• Database: DM8 / MySQL, PostgreSQL
• Cache & Message Queue: Redis, RabbitMQ
• File & Storage: MongoDB, Hadoop, kkFileView
• Scheduler: DolphinScheduler

Start the middleware using Docker Compose:

# Local source mode from the Docker Compose deployment documentation
docker-compose --profile local up -d

👉 Refer to Docker Compose Deployment Documentation

5. Start the Backend

5.1 Environment Preparation

• JDK = 1.8
• Maven >= 3.6

5.2 Launch Project in IDEA

Open the project using IntelliJ IDEA.

5.3 Build the Project

mvn clean -DskipTests=true install

5.4 Essential Configurations

All configurations below must be modified in the application-dev.yml file.

5.4.1 qdata-server Configuration

• Modify RabbitMQ connection info. Edit qdata-server/src/main/resources/application-dev.yml:

spring:
  rabbitmq:
    host: IP
    port: Port
    username: Username
    password: Password

• Modify database connection info. Edit qdata-server/src/main/resources/application-dev.yml:

# Primary data source selection
datasource:
  type: mysql # mysql or dm8; modify corresponding section below accordingly

# MySQL Configuration
mysql:
  # JDBC driver class name
  driver-class-name: com.mysql.cj.jdbc.Driver
  # Primary database JDBC URL
  url: jdbc:mysql://IP:Port/qdata?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8
  # Primary database username
  username: Username
  # Primary database password
  password: Password

# DM8 (Dameng) Configuration
dm8:
  # JDBC driver class name
  driver-class-name: dm.jdbc.driver.DmDriver
  # Primary database JDBC URL
  url: jdbc:dm://IP:Port/QDATA?STU&zeroDateTimeBehavior=convertToNull&useUnicode=true&characterEncoding=utf-8&schema=QDATA&serverTimezone=Asia/Shanghai
  # Primary database username
  username: Username
  # Primary database password
  password: Password

• Modify scheduler-related configurations. Edit qdata-server/src/main/resources/application-dev.yml:

# Scheduler configurations
ds:
  # Scheduler API base URL
  base_url: http://IP:12345/dolphinscheduler
  # HTTP callback URL for data quality service component
  quality_url: http://127.0.0.1:8083/quality/qualityTaskExecutor/runExecuteTask
  # Data quality task project code in DolphinScheduler
  http_quality_projectCode: 134799536571008
  # Scheduler token (can be customized)
  token: b07f0e57c4818043a57ba05a28da291a
  # Spark configurations
  spark:
    # Spark master URL
    master_url: spark://IP:7077
    # JAR path in DolphinScheduler
    main_jar: file:/dolphinscheduler/default/resources/spark-jar/qdata-etl-3.8.8.jar
    # JAR main entry class
    main_class: tech.qiantong.qdata.spark.etl.EtlApplication
  # HDFS configurations
  hdfs:
    # NameNode RPC address (client entry point for HDFS)
    url: hdfs://IP:Port

• Modify data quality service configurations. Edit qdata-server/src/main/resources/application-dev.yml:

path:
  # Data quality service base URL
  quality_url: http://127.0.0.1:8083/quality/qualityTaskExecutor

5.4.2 qdata-quality Configuration

• Modify RabbitMQ connection info. Edit qdata-quality/src/main/resources/application-dev.yml:

spring:
  rabbitmq:
    host: IP
    port: Port
    username: Username
    password: Password

• Modify MongoDB connection info. Edit qdata-quality/src/main/resources/application-dev.yml:

spring:
  data:
    mongodb:
      # Connection URI
      uri: mongodb://Username:Password@IP:Port/data?authSource=admin
      field-naming-strategy: org.springframework.data.mapping.model.SnakeCaseFieldNamingStrategy
      print: true
      slowQuery: true
      slowTime: 1000

• Modify database connection info. Edit qdata-quality/src/main/resources/application-dev.yml:

# Primary data source selection
datasource:
  type: mysql # mysql or dm8; modify corresponding section below accordingly

# MySQL Configuration
mysql:
  # JDBC driver class name
  driver-class-name: com.mysql.cj.jdbc.Driver
  # Primary database JDBC URL
  url: jdbc:mysql://IP:Port/qdata?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8
  # Primary database username
  username: Username
  # Primary database password
  password: Password

# DM8 (Dameng) Configuration
dm8:
  # JDBC driver class name
  driver-class-name: dm.jdbc.driver.DmDriver
  # Primary database JDBC URL
  url: jdbc:dm://IP:Port/QDATA?STU&zeroDateTimeBehavior=convertToNull&useUnicode=true&characterEncoding=utf-8&schema=QDATA&serverTimezone=Asia/Shanghai
  # Primary database username
  username: Username
  # Primary database password
  password: Password

5.4.3 Start the qdata-server Service

Run qdata-server/src/main/java/tech/qiantong/qdata/server/QDataApplication.java.

5.4.4 Start the qdata-quality Service

Run qdata-quality/src/main/java/tech/qiantong/qdata/quality/QualityApplication.java.

6. Start the Frontend

6.1 Environment Preparation

• Node.js >= 18.0
• yarn >= v1.22.22

6.2 Start the Frontend Service

# Enter frontend directory
cd qdata-ui
# Install dependencies
yarn install
# Start service
yarn run dev

6.3 Reset Admin Password

Execute the following SQL in the database:

UPDATE SYSTEM_USER
SET PASSWORD='$2a$10$3dzj0nGSmlNjPw4iuZNHse5.lxrNzumTCQ8aEaV7nNEtn2pXmmimW'
WHERE USER_ID=1;

6.4 Access qData

Open the following URL in your browser: http://localhost:81
Default username/password: admin / qData123

If the login page displays correctly, you can log in successfully, and menus and pages appear normally, your environment has been set up successfully.