我是微软Dynamics 365 & Power Platform方面的工程师罗勇,也是2015年7月到2018年6月连续三年Dynamics CRM/Business Solutions方面的微软最有价值专家(Microsoft MVP),欢迎关注我的微信公众号 MSFTDynamics365erLuoYong ,回复351或者20190816可方便获取本文,同时可以在第一间得到我发布的最新博文信息,follow me!
利用标准的导入功能来导入数据是基本操作,操作起来简单引用,比较受用户喜欢,而且导入速度也可以,特别是并行导入的情况下。
最近碰到用CSV来导入数据,在导入之前将数据拆分成多个CSV文件,每个文件不超过8M,这些CSV文件打包成Zip文件后不超过32MB,上传后是差不多一个一个处理,并行处理没有那么明显了。如下图,我最开始用一个csv上传了一个文件进行导入,后面我用一个zip包上传了5个csv文件来导入,这个zip文件中的5个文件要会等到第一个处理完毕后再开始。然后zip中的文件有些并行处理,但是看起来不明显,下图可以看到 data02.csv 和 data03.csv都处于解析阶段。
我的印象中,Dynamics 365 CRM V8.X是可以并行导入的,而且处理速度挺快的,是Dynamics 365 Customer Engagement V9.X开始因为调用API的限制等并行不是那么厉害了吗?如果是,谁看到哪儿有说明请告知我,我更新下本博文,可由什么参数可以更改此类设置。
我查看了导入相关的部署属性,参考 Update deployment configuration settings ,查看的命令参考如下:
Add-PSSnapin Microsoft.Crm.PowerShell
Get-CrmSetting ImportSettings
我的环境显示结果如下,这些值应该是默认值。
如果是一个文件一个文件导入,速度不会很快,我们可以用程序来导入,多线程加上ExecuteMultipleRequest消息速度还是可以的。
我这里贴一个导入的代码:
using Microsoft.Xrm.Sdk; using Microsoft.Xrm.Sdk.Client; using Microsoft.Xrm.Sdk.Messages; using System; using System.Configuration; using System.IO; using System.ServiceModel; using System.Threading; namespace BulkImportRecords { class Program { public static IServiceManagement<IOrganizationService> sm; public static AuthenticationCredentials authCredentials; static int importsequencenumberstartat = Convert.ToInt32(ConfigurationManager.AppSettings["importsequencenumberstartat"]); static int threadcount = Convert.ToInt32(ConfigurationManager.AppSettings["threadcount"]); static void Main(string[] args) { sm = ServiceConfigurationFactory.CreateManagement<IOrganizationService>(new Uri(ConfigurationManager.AppSettings["orgUrl"])); authCredentials = new AuthenticationCredentials(); authCredentials.ClientCredentials.UserName.UserName = ConfigurationManager.AppSettings["userName"]; authCredentials.ClientCredentials.UserName.Password = ConfigurationManager.AppSettings["passWord"]; authCredentials = sm.Authenticate(authCredentials); try { for (var i = 0; i < threadcount; i++) { Thread newThread = new Thread(new ParameterizedThreadStart(Work)); newThread.Start(i); } Console.ReadKey(); } catch (FaultException ex) { Console.WriteLine("程序出现异常:ex.Message=" + ex.Message); Console.ReadKey(); } } static void Work(object data) { try { Console.WriteLine("线程开始" + DateTime.Now.ToLongTimeString() + ";线程ID:" + Thread.CurrentThread.ManagedThreadId + ";接收的参数值为:" + data.ToString()); int importsequencenumber = importsequencenumberstartat + Convert.ToInt32(data); OrganizationServiceProxy orgSvc = new OrganizationServiceProxy(sm, authCredentials.ClientCredentials); //OrganizationServiceProxy orgSvc = new OrganizationServiceProxy(sm, authCredentials.SecurityTokenResponse); //ManagedTokenOrganizationServiceProxy orgSvc = new ManagedTokenOrganizationServiceProxy(sm, authCredentials.ClientCredentials); string strReadFilePath = ConfigurationManager.AppSettings["filename"]; int i = 0; int j = 1; int z = 1; ExecuteMultipleRequest multiReqs = new ExecuteMultipleRequest() { Settings = new ExecuteMultipleSettings() { ContinueOnError = true, ReturnResponses = false }, Requests = new OrganizationRequestCollection() }; using (StreamReader srReadFile = new StreamReader(string.Format(strReadFilePath, (Convert.ToInt32(data) + 1).ToString("00")))) { while (!srReadFile.EndOfStream) { string strReadLine = srReadFile.ReadLine(); //读取每行数据 if (i != 0)//如果第一行包括标题的话要过滤掉 { var arrLine = strReadLine.Split(‘,‘); CreateRequest req = new CreateRequest(); var createEntity = new Entity("ly_test"); createEntity["ly_name"] = arrLine[0]; createEntity["ly_singletext1"] = arrLine[1]; createEntity["ly_singletext2"] = arrLine[2]; createEntity["ly_singletext3"] = arrLine[3]; createEntity["importsequencenumber"] = Convert.ToInt32(importsequencenumber); req.Target = createEntity; if (j <= 1000) { multiReqs.Requests.Add(req); } else { multiReqs.Requests = new OrganizationRequestCollection(); multiReqs.Requests.Add(req); j = 1; } if (j == 1000) { orgSvc.Execute(multiReqs); Console.WriteLine("线程:" + Thread.CurrentThread.ManagedThreadId + "-导入完毕" + z + "条" + DateTime.Now.ToString()); } z++; j++; } i++; } } orgSvc.Execute(multiReqs); Console.WriteLine("线程结束" + DateTime.Now.ToLongTimeString() + ";线程ID:" + Thread.CurrentThread.ManagedThreadId); } catch(FaultException<Microsoft.Xrm.Sdk.OrganizationServiceFault> ex) { Console.WriteLine("执行遇到异常:" + ex.Detail.ErrorCode + ex.Message + ex.StackTrace); } catch (Exception e) { Console.WriteLine("执行遇到异常:" + e.Message + e.StackTrace); } } } }
这个程序配合的配置文件如下:
<?xml version="1.0" encoding="utf-8"?> <configuration> <startup> <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.2"/> </startup> <appSettings> <add key="userName" value="crmadmin@luoyong.me"/> <add key="passWord" value="Pass"/> <add key="orgUrl" value="https://demo.luoyong.me/XRMServices/2011/Organization.svc"/> <add key="filename" value="D:\dataimport\data{0}.csv"/> <add key="importsequencenumberstartat" value="1000000"/> <add key="threadcount" value="10"/> </appSettings> </configuration>
Dynamics 365 Customer Engagement的标准导入不支持并行导入了吗?
原文:https://www.cnblogs.com/luoyong0201/p/Dynamics_365_CE_Import_Data.html