MogileFS的安装与配置
================================================================================
概述：
================================================================================
1.perl模块的安装方式：
★online：
yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes
执行命令 ： perl -MCPAN -e shell 进入 perl 依赖包安装模式 （或执行 cpan -i module_name::name）
cpan install App::cpanminus
cpan install MogileFS::Server
cpan install MogileFS::Utils
cpan install IO::AIO
cpan install IO::WrapTie
cpan install Danga::Socket★offline：
$ perl Makefile.PL
$ make
$ sudo make install
★rpm
☉MogileFS安装：
tracker：
$ yum install MogileFS-Server-VERSION.rpm MogileFS-Server-mogilefsd-VERSION.rpm
mogstored：
$ yum install perl-IO-AIO
$ yum install MogileFS-Server-VERSION.rpm MogileFS-Server-mogstored-VERSION.rpm
实验：MofileFS的安装和配置
环境描述：
两台CentOS 7的虚拟主机node1和node2，node1即作为tracker又作为mogstore存储节点，同时还是mysql服务器，node2节点只部署mogstore；
在node1服务器上部署tracker管理并监控节点。
实验步骤：
1.首先在noge1上使用CPAN工具在线安装perl模块及mogilefs
[root@centos7 ~]# yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes [root@centos7 ~]# perl -MCPAN -e shell cpan >install App::cpanminus cpan >install MogileFS::Server cpan >install MogileFS::Utils cpan >install IO::AIO cpan >install IO::WrapTie cpan >install Danga::Socket
2.在node1上安装MySQL（略），创建数据库，添加管理用户
[root@centos7 ~]# mysql -ptaoxiu Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 11 Server version: 5.5.44-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement. MariaDB [(none)]> create database mogfsdb; # 创建数据库 MariaDB [(none)]> grant all privileges on mogfsdb.* to ‘moguser‘@‘127.0.0.1‘ identified by ‘mogpass‘; MariaDB [(none)]> grant all privileges on mogfsdb.* to ‘moguser‘@‘localhost‘ identified by ‘mogpass‘; MariaDB [(none)]> flush privileges;
3.初始化数据库
[root@centos7 ~]# mogdbsetup --dbhost=‘127.0.0.1‘ --dbname=mogfsdb --dbuser=moguser --dbpass=mogpass --type=MySQL This will attempt to setup or upgrade your MogileFS database. It won‘t destroy existing data. Run with --help for more information. Run with --yes to shut up these prompts. Continue? [N/y]: y
查看数据库如下：
[root@centos7 ~]# mysql -ptaoxiu Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 5 Server version: 5.5.44-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement. MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | Syslog | | hellodb | | jingdong | | mogfsdb | | mysql | | ultrax | | zabbix | +--------------------+ 8 rows in set (0.12 sec) MariaDB [(none)]> use mogfsdb; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed MariaDB [mogfsdb]> show tables; 生成的表 +----------------------+ | Tables_in_mogfsdb | +----------------------+ | checksum | | class | | device | | domain | | file | | file_on | | file_on_corrupt | | file_to_delete | | file_to_delete2 | | file_to_delete_later | | file_to_queue | | file_to_replicate | | fsck_log | | host | | server_settings | | tempfile | | unreachable_fids | +----------------------+ 17 rows in set (0.00 sec) MariaDB [mogfsdb]> exit
配置tracker节点，
1.在/etc目录下创建mogilefs目录，并提供配置文件mogilefsd.conf，如下：
[root@centos7 ~]# mkdir /etc/mogilefs [root@centos7 ~]# vim /etc/mogilefs/mogilefsd.conf # Enable daemon mode to work in background and use syslog daemonize = 1 # Where to store the pid of the daemon (must be the same in the init script) pidfile = /var/run/mogilefsd/mogilefsd.pid # Database connection information db_dsn = DBI:mysql:mogfsdb:host=127.0.0.1 db_user = moguser db_pass = mogpass # IP:PORT to listen on for mogilefs client requests listen = 0.0.0.0:7001 # Optional, if you don‘t define the port above. conf_port = 7001 # Number of query workers to start by default. query_jobs = 10 # Number of delete workers to start by default. delete_jobs = 1 # Number of replicate workers to start by default. replicate_jobs = 5 # Number of reaper workers to start by default. # (you don‘t usually need to increase this) reaper_jobs = 1 # Number of fsck workers to start by default. # (these can cause a lot of load when fsck‘ing) #fsck_jobs = 1 # Minimum amount of space to reserve in megabytes # default: 100 # Consider setting this to be larger than the largest file you # would normally be uploading. #min_free_space = 200 # Number of seconds to wait for a storage node to respond. # default: 2 # Keep this low, so busy storage nodes are quickly ignored. #node_timeout = 2 # Number of seconds to wait to connect to a storage node. # default: 2 # Keep this low so overloaded nodes get skipped. #conn_timeout = 2 # Allow replication to use the secondary node get port, # if you have apache or similar configured for GET‘s #repl_use_get_port = 1
2.创建pid文件目录并赋予权限，然后启动服务。
[root@centos7 ~]# mkdir /var/run/mogilefsd/ -p [root@centos7 ~]# chown -R mogilefs.mogilefs /var/run/mogilefsd/ # mogilefsd不能以root身份运行，必须用其他用户运行，为此添加mogilefs用户，然后启动服务 [root@centos7 ~]# useradd -r mogilefs [root@centos7 mogilefs]# su - mogilefs -c "mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon" su: warning: cannot change directory to /home/mogilefs: No such file or directory # 查看端口7001 [root@centos7 mogilefs]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 25 *:514 *:* LISTEN 0 50 *:3306 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 127.0.0.1:631 *:* LISTEN 0 128 *:7001 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 127.0.0.1:6010 *:* LISTEN 0 25 :::514 :::* LISTEN 0 128 :::22 :::* LISTEN 0 128 ::1:631 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 128 ::1:6010 :::*
接下来配置 mogstore节点，
1.编辑配置文件 vim /etc/mogilefs/mogstored.conf如下：
[root@centos7 ~]# vim /etc/mogilefs/mogstored.conf maxconns = 10000 # 最大并发连接数 httplisten = 0.0.0.0:7500 # 监听端口 mgmtlisten = 0.0.0.0:7501 # 管理端口，探测mogstore的健康状态 docroot = /data/mogstored # 数据存放位置
2.提供数据存放目录/data/mogstored并修改属主属组为mogilefs
[root@centos7 ~]# mkdir -pv /data/mogstored
# 添加磁盘DEV（生产环境中要把磁盘设备挂载到此目录下，这里为了测试创建目录DEV）
[root@centos7 ~]# mkdir /data/mogstored/dev{1,2}
# 修改属主属组为mogilefs
[root@centos7 ~]# chown -R mogilefs:mogilefs /data/mogstored/
[root@centos7 mogstored]# ll
total 0
drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:52 DEV1
drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:52 DEV23.启动服务，并查看端口7500,7501
[root@centos7 mogstored]# su - mogilefs -c "mogstored -c /etc/mogilefs/mogstored.conf --daemon" su: warning: cannot change directory to /home/mogilefs: No such file or directory [root@centos7 mogstored]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 25 *:514 *:* LISTEN 0 50 *:3306 *:* LISTEN 0 128 *:7500 *:* LISTEN 0 128 *:7501 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 127.0.0.1:631 *:* LISTEN 0 128 *:7001 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 127.0.0.1:6010 *:* LISTEN 0 25 :::514 :::* LISTEN 0 128 :::22 :::* LISTEN 0 128 ::1:631 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 128 ::1:6010 :::*
在节点node2上同样配置mogstore存储节点，如下：
1.同样适用CPAN在线安装perl相关模块和mogilefs，存储节点要解决依赖关系，安装perl-IO-AIO
[root@centos7 ~]# yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes [root@centos7 ~]# perl -MCPAN -e shell cpan >install App::cpanminus cpan >install MogileFS::Server cpan >install MogileFS::Utils cpan >install IO::AIO cpan >install IO::WrapTie cpan >install Danga::Socket [root@centos7 ~]# yum -y install perl-IO-AIO
2.在/etc下创建目录，并提供mogstore的配置文件mogstored.conf
[root@centos7 ~]# mkdir /etc/mogilefs [root@centos7 ~]# vim /etc/mogilefs/mogstored.conf maxconns = 10000 httplisten = 0.0.0.0:7500 mgmtlisten = 0.0.0.0:7501 docroot = /data/mogstored
3.提供数据存放目录/data/mogstored并修改属主属组为mogilefs
[root@centos7 ~]# mkdir -pv /data/mogstored/dev{3,4} # 两个节点设备不能相同，且为小写
mkdir: created directory ‘/data/mogstored’
mkdir: created directory ‘/data/mogstored/DEV1’
mkdir: created directory ‘/data/mogstored/DEV2’
[root@centos7 ~]# chown -R mogilefs:mogilefs /data/mogstored/
[root@centos7 ~]# cd /data/mogstored/
[root@centos7 mogstored]# ll
total 0
drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:22 DEV1
drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:22 DEV24.创建mogilefs用户，启动服务，查看端口
[root@centos7 ~]# useradd -r mogilefs [root@centos7 ~]# su - mogilefs -c "mogstored -c /etc/mogilefs/mogstored.conf --daemon" su: warning: cannot change directory to /home/mogilefs: No such file or directory [root@centos7 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 50 *:3306 *:* LISTEN 0 128 *:7500 *:* LISTEN 0 128 *:7501 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 127.0.0.1:631 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 127.0.0.1:6011 *:* LISTEN 0 128 :::22 :::* LISTEN 0 128 ::1:631 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 128 ::1:6011 :::*
如上，我们就有了一个tracker和连个mogstore，接下来我们可以在控制节点上使用MogileFS-Utils，进行添加host，device，domain，class等管理操作。
1.使用mogadm命令添加存储节点mogstore
[root@centos7 ~]# mogadm host add node1 --ip=192.168.1.112 --status=alive --port=7500 [root@centos7 ~]# mogadm host add node2 --ip=192.168.1.113 --status=alive --port=7500 [root@centos7 ~]# mogadm host list node1 [1]: alive IP: 192.168.1.112:7500 node2 [2]: alive IP: 192.168.1.113:7500
2.使用mogadm命令添加存储节点device，注意，前面挂载的设备必须为小写dev且个存储节点名称不能相同
[root@centos7 ~]# mogadm device add node1 1 --status=alive [root@centos7 ~]# mogadm device add node1 2 --status=alive [root@centos7 ~]# mogadm device add node2 3 --status=alive [root@centos7 ~]# mogadm device add node2 4 --status=alive [root@centos7 ~]# mogadm device list node1 [1]: alive used(G) free(G) total(G) weight(%) dev1: alive 3.674 36.307 39.980 100 dev2: alive 3.674 36.307 39.980 100 node2 [2]: alive used(G) free(G) total(G) weight(%) dev3: alive 0.398 39.582 39.980 100 dev4: alive 0.398 39.582 39.980 100
3.添加domain域或者类class
[root@centos7 ~]# mogadm domain list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- [root@centos7 ~]# mogadm domain add files [root@centos7 ~]# mogadm domain add imgs [root@centos7 ~]# mogadm domain list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- files default 2 MultipleHosts() NONE imgs default 2 MultipleHosts() NONE # mindevcount:默认最小设备数为2，即默认创建了两个副本 # replpolicy：复制策略为多主机复制 # hashtype：哈希类型，NOME表示不自定义，使用默认值 [root@centos7 ~]# mogadm class list domain class mindevcount replpolicy hashtype -------------------- -------------------- ------------- ------------ ------- files default 2 MultipleHosts() NONE imgs default 2 MultipleHosts() NONE
4.查看检测状态
[root@centos7 ~]# mogadm check Checking trackers... # tracker识别OK 127.0.0.1:7001 ... OK Checking hosts... # 主机识别OK [ 1] node1 ... OK [ 2] node2 ... OK Checking devices... # 存储设备识别OK host device size(G) used(G) free(G) use% ob state I/O% ---- ------------ ---------- ---------- ---------- ------ ---------- ----- [ 1] dev1 39.980 3.675 36.306 9.19% writeable 0.0 [ 1] dev2 39.980 3.675 36.306 9.19% writeable 0.0 [ 2] dev3 39.980 0.400 39.581 1.00% writeable 0.0 [ 2] dev4 39.980 0.400 39.581 1.00% writeable 0.0 ---- ------------ ---------- ---------- ---------- ------ total: 159.922 8.149 151.773 5.10%
5.使用mogupload命令上传文件
[root@centos7 ~]# mogupload --help # 上传文件 Usage: /usr/local/bin/mogupload --trackers=host --domain=foo --key=‘/hello.jpg‘ --file=‘./hello.jpg‘ [root@centos7 ~]# mogupload --trackers=192.168.1.112 --domain=files --key=‘/fstab.txt‘ --file=‘/etc/fstab‘ [root@centos7 ~]# mogupload --trackers=192.168.1.112 --domain=imgs --key=‘/man.jpg‘ --file=‘/root/man.jpg‘ [root@centos7 ~]# moglistkeys --help # 列出文件 Usage: /usr/local/bin/moglistkeys --trackers=host --domain=foo --key_prefix=‘bar/‘ [root@centos7 ~]# moglistkeys --trackers=192.168.1.112 --domain=files /fstab.txt [root@centos7 ~]# moglistkeys --trackers=192.168.1.112 --domain=imgs /man.jpg
6.使用mogfileinfo查看文件信息，如下：
[root@centos7 ~]# mogfileinfo --trackers=192.168.1.112 --domain=files --key=‘/fstab.txt‘ - file: /fstab.txt class: default devcount: 1 domain: files fid: 2 key: /fstab.txt length: 690 - # 可通过此URL访问到相应的资源 [root@centos7 ~]# mogfileinfo --trackers=192.168.1.112 --domain=imgs --key=‘/man.jpg‘ - file: /man.jpg class: default devcount: 1 domain: imgs fid: 3 key: /man.jpg length: 3401017 - http://192.168.1.112:7500/dev2/0/000/000/0000000003.fid
通过URL访问其资源如下：
7.使用mogstats指明连接的数据库，用户和密码，查看状态，如下：
[root@centos7 ~]# mogstats --db_dsn="DBI:mysql:mogfsdb:host=127.0.0.1" --db_user="moguser" --db_pass="mogpass" --stats="all" Fetching statistics... (all) Statistics for devices... device host files status ---------- ---------------- ------------ ---------- dev1 node1 1 alive dev2 node1 1 alive ---------- ---------------- ------------ ---------- Statistics for file ids... Max file id: 3 Statistics for files... domain class files size (m) fullsize (m) -------------------- ----------- ---------- ----------- ------------- files default 1 0 0 imgs default 1 3 3 -------------------- ----------- ---------- ----------- ------------- Statistics for replication... domain class devcount files -------------------- ----------- ---------- ---------- files default 1 1 imgs default 1 1 -------------------- ----------- ---------- ---------- Statistics for replication queue... status count -------------------- ------------ deferred 2 -------------------- ------------ Statistics for delete queue... status count -------------------- ------------ -------------------- ------------ Statistics for general queues... queue status count --------------- -------------------- ------------ --------------- -------------------- ------------ done
本文出自 “逐梦小涛” 博客，请务必保留此出处http://1992tao.blog.51cto.com/11606804/1889437
原文：http://1992tao.blog.51cto.com/11606804/1889437