首页 > 其他 > 详细

分布式文件系统FastDFS实战

时间:2017-01-17 22:07:37      阅读:285      评论:0      收藏:0      [点我收藏+]

(4kb-500MB之间)

角色:

    client

    tracker群

    storage群


环境:两台 分别安装    tracker群    storage群

Tracker Server:

跟踪服务器,主要做调度工作,在访问中起负载均衡的作用。在内存中记录集群中group和storage的状态信息,是连接client和storage server的枢纽,因为相关信息全部在内存中,tracker server的性能非常高,一个较大的集群(上百group)3台即可。

Storage Server:

存储服务器,文件和文件属性(meta data)都保存在上面。

https://github.com/happyfish100

安装依赖:

git clone https://github.com/happyfish100/libfastcommon.git
cd libfastcommon/
./make.sh
./make.sh install

安装软件:

https://github.com/happyfish100/fastdfs/releases
tar zxf fastdfs-5.05.tar.gz 
cd fastdfs-5.05
./make.sh 
./make.sh install

[root@node1-197 fastdfs-5.05]# ll /etc/fdfs/

总用量 20

-rw-r--r-- 1 root root 1461 8月  24 14:05 client.conf.sample
-rw-r--r-- 1 root root 7829 8月  24 14:05 storage.conf.sample
-rw-r--r-- 1 root root 7102 8月  24 14:05 tracker.conf.sample
[root@node1-197 fastdfs-5.05]# ll /usr/bin/fdfs_
fdfs_appender_test    fdfs_delete_file      fdfs_storaged         fdfs_upload_appender
fdfs_appender_test1   fdfs_download_file    fdfs_test             fdfs_upload_file
fdfs_append_file      fdfs_file_info        fdfs_test1            
fdfs_crc32            fdfs_monitor          fdfs_trackerd         
[root@node1-197 fastdfs-5.05]# ll /etc/init.d/fdfs_
fdfs_storaged  fdfs_trackerd  
[root@node1-197 fastdfs-5.05]# sed -i ‘s#/usr/local/bin/#/usr/bin/#‘ /etc/init.d/fdfs_*
cd /etc/fdfs/
cp tracker.conf.sample tracker.conf
vim tracker.conf

创建目录:

[root@centos6-vm-53 fdfs]# tree /data/

/data/
├── fdfs_storage
│   ├── base
│   └── store
└── fdfs_tracker
    ├── data
    │   ├── fdfs_trackerd.pid
    │   └── storage_changelog.dat
    └── logs
        └── trackerd.log
/etc/init.d/fdfs_trackerd start
lsof -i:22122
lsof -i:23000

>-----------------------------------------------------

[root@node1-197 ~]# tree /etc/fdfs/

/etc/fdfs/
|-- client.conf
|-- client.conf.sample
|-- storage.conf
|-- storage.conf.sample
|-- tracker.conf
`-- tracker.conf.sample
0 directories, 6 files

>-----------------------------------------------------

存储节点:

/etc/fdfs
cp storage.conf.sample storage.conf
修改内容:
base_path=/data/fdfs_storage/base
store_path0=/data/fdfs_storage/store
tracker_server=192.168.1.61:22122
tracker_server=192.168.1.53:22122

客户端操作:

 cp client.conf.sample client.conf
base_path=/tmp
tracker_server=192.168.1.61:22122
tracker_server=192.168.1.53:22122

[root@node1-197 /]# fdfs_upload_file /etc/fdfs/client.conf /etc/passwd
group1/M00/00/00/wKjrxVXawRCAHqZqAAAKU60-8sE9427256
[root@node1-197 00]# pwd
/data/fdfs_storage/store/data/00/00
[root@node1-197 00]# ls
wKjrxVXawRCAHqZqAAAKU60-8sE9427256
[root@node1-197 ~]# md5sum  wKjrxVXawRCAHqZqAAAKU60-8sE9427256
78d23b6ae5271f6ecfd0e51d2dc1d164  wKjrxVXawRCAHqZqAAAKU60-8sE9427256
[root@node1-197 ~]# md5sum /etc/passwd
78d23b6ae5271f6ecfd0e51d2dc1d164  /etc/passwd
fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/wKjrxVXawRCAHqZqAAAKU60-8sE9427256
[root@node1-197 ~]# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/wKjrxVXawRCAHqZqAAAKU60-8sE9427256    
source storage id: 0
source ip address: 192.168.235.197
file create timestamp: 2015-08-24 15:00:32
file size: 2643
file crc32: 2906583745 (0xAD3EF2C1)
[root@node1-197 ~]# fdfs_delete_file /etc/fdfs/client.conf group1/M00/00/00/wKjrxVXawRCAHqZqAAAKU60-8sE9427256

上传可追加的文件:

fdfs_upload_appender /etc/fdfs/client.conf xx.txt

下载后追加:

fdfs_append_file

命令:

[root@node1-197 ~]# fdfs_
fdfs_appender_test    fdfs_delete_file      fdfs_storaged         fdfs_upload_appender
fdfs_appender_test1   fdfs_download_file    fdfs_test             fdfs_upload_file
fdfs_append_file      fdfs_file_info        fdfs_test1            
fdfs_crc32            fdfs_monitor          fdfs_trackerd     
[root@node1-197 ~]# fdfs_monitor /etc/fdfs/storage.conf

php客户端:

cd /usr/local/src/fastdfs-5.05/php_client/
/usr/local/php56/bin/phpize 
./configure --with-php-config=/usr/local/php56/bin/php-config 
make && make install

提示:

Installing shared extensions:     /usr/local/php56/lib/php/extensions/no-debug-non-zts-20131226/


[root@centos6-vm-53 php_client]# cat fastdfs_client.ini >> /usr/local/php56/etc/php.i


测试:

[root@centos6-vm-53 php_client]# /usr/local/php56/bin/php fastdfs_test.php


java客户端:

cd /usr/local/src/
git clone https://github.com/happyfish100/fastdfs-client-java.git
cd fastdfs-client-java/src

执行ant:


[root@zhong-61 src]# ant

Buildfile: build.xml


init:


compile:

    [mkdir] Created dir: /usr/local/src/fastdfs-client-java/src/build/classes

    [javac] Compiling 32 source files to /usr/local/src/fastdfs-client-java/src/build/classes

    [javac] Note: Some input files use unchecked or unsafe operations.

    [javac] Note: Recompile with -Xlint:unchecked for details.


jar:

      [jar] Building jar: /usr/local/src/fastdfs-client-java/src/build/fastdfs_client.jar


BUILD SUCCESSFUL

Total time: 5 seconds


用法:

https://github.com/happyfish100/fastdfs-client-java/tree/master/src
[root@zhong-61 src]# java -cp /usr/local/src/fastdfs-client-java/src/build/fastdfs_client.jar org.csource.fastdfs.test.TestClient /etc/fdfs/client.conf /etc/passwd
[root@zhong-61 src]# java -cp /usr/local/src/fastdfs-client-java/src/build/fastdfs_client.jar org.csource.fastdfs.test.Monitor /etc/fdfs/client.conf

nginx模块:

[root@centos6-vm-53 src]# git clone https://github.com/happyfish100/fastdfs-nginx-module.git
[root@centos6-vm-53 nginx-1.9.9]# ./configure --prefix=/usr/local/nginx --without-http_memcached_module --user=nginx --group=nginx --with-http_stub_status_module --with-pcre --with-http_ssl_module --add-module=/usr/local/src/fastdfs-nginx-module/src/
make && make install

[root@centos6-vm-53 src]# pwd
/usr/local/src/fastdfs-nginx-module/src
[root@centos6-vm-53 src]# cp mod_fastdfs.conf /etc/fdfs/
[root@centos6-vm-53 src]# cd /usr/local/src/fastdfs-5.05/conf/
[root@centos6-vm-53 conf]# ls
anti-steal.jpg  client.conf  http.conf  mime.types  storage.conf  storage_ids.conf  tracker.conf
[root@centos6-vm-53 conf]# cp anti-steal.jpg http.conf mime.types /etc/fdfs/
[root@centos6-vm-53 conf]# touch /var/log/mod_fastdfs.log
[root@centos6-vm-53 conf]# chown nginx.nginx /var/log/mod_fastdfs.log

nginx  server增加:

    location /group/M00 {
        root /data/fdfs_storage/store/;
        ngx_fastdfs_module;
    }


[root@centos6-vm-53 conf.d]# vim /etc/fdfs/mod_fastdfs.conf 
tracker_server=192.168.1.61:22122
tracker_server=192.168.1.53:22122
url_have_group_name = true
store_path0=/data/fdfs_storage/store
log_filename=/var/log/mod_fastdfs.log

重启nginx,访问测试:

http://192.168.1.53:800/group1/M00/00/00/wKgBNVaCV8GAc0CjAAAgVBESXKk681.png

分布式文件系统FastDFS实战

原文:http://bobo365.blog.51cto.com/1035992/1892623

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!