2.793

                    2018影響因子

                    (CJCR)

                    • 中文核心
                    • EI
                    • 中國科技核心
                    • Scopus
                    • CSCD
                    • 英國科學文摘

                    留言板

                    尊敬的讀者、作者、審稿人, 關于本刊的投稿、審稿、編輯和出版的任何問題, 您可以本頁添加留言。我們將盡快給您答復。謝謝您的支持!

                    姓名
                    郵箱
                    手機號碼
                    標題
                    留言內容
                    驗證碼

                    一種隨機配置網絡的模型與數據混合并行學習方法

                    代偉 李德鵬 楊春雨 馬小平

                    代偉, 李德鵬, 楊春雨, 馬小平. 一種隨機配置網絡的模型與數據混合并行學習方法. 自動化學報, 2019, 45(x): 1?11 doi: 10.16383/j.aas.c190411
                    引用本文: 代偉, 李德鵬, 楊春雨, 馬小平. 一種隨機配置網絡的模型與數據混合并行學習方法. 自動化學報, 2019, 45(x): 1?11 doi: 10.16383/j.aas.c190411
                    Dai Wei, Li De-Peng, Yang Chun-Yu, Ma Xiao-Ping. A model and data hybrid parallel learning method for stochastic configuration networks. Acta Automatica Sinica, 2019, 45(x): 1?11 doi: 10.16383/j.aas.c190411
                    Citation: Dai Wei, Li De-Peng, Yang Chun-Yu, Ma Xiao-Ping. A model and data hybrid parallel learning method for stochastic configuration networks. Acta Automatica Sinica, 2019, 45(x): 1?11 doi: 10.16383/j.aas.c190411

                    一種隨機配置網絡的模型與數據混合并行學習方法

                    doi: 10.16383/j.aas.c190411
                    基金項目: 國家自然科學基金(61603393, 61973306), 江蘇省自然科學基金(BK20160275), 中國博士后科學基金(2018T110571), 流程工業綜合自動化國家重點實驗室開放基金資助(PAL-N201706)
                    詳細信息
                      作者簡介:

                      代偉:中國礦業大學信息與控制工程學院副教授. 主要研究方向為復雜工業過程建模、運行優化與控制. 本文通信作者. E-mail: weidai@cumt.edu.cn

                      李德鵬:中國礦業大學信息與控制工程學院碩士研究生. 主要研究方向為數據驅動建模、機器學習算法. E-mail:dpli@cumt.edu.cn

                      楊春雨:中國礦業大學信息與控制工程學院教授. 于2009年獲得東北大學博士學位. 主要研究方向為廣義系統和魯棒控制. E-mail: chunyuyang@cumt.edu.cn

                      馬小平:中國礦業大學信息與控制工程學院教授. 主要研究方向為過程控制、網絡控制與故障診斷. E-mail: xpma@cumt.edu.cn

                    A Model and Data Hybrid Parallel Learning Method for Stochastic Configuration Networks

                    Funds: Supported by National Natural Science Foundation of China(61603393, 61973306), Natural Science Foundation of Jiangsu Provinces(BK20160275), the Postdoctoral Science Foundation of China(2018T110571), State Key Laboratory of Synthetical Automation for Process Industries (PAL-N201706)
                    • 摘要: 隨機配置網絡(Stochastic configuration networks, SCNs)在增量構建過程引入監督機制來分配隱含層參數以確保其無限逼近特性, 具有易于實現、收斂速度快、泛化性能好等優點. 然而, 隨著數據量的不斷擴大, SCNs的建模任務面臨一定的挑戰性. 為了提高神經網絡算法在大數據建模中的綜合性能, 本文提出了一種混合并行隨機配置網絡(Hybrid parallel stochastic configuration networks, HPSCNs)架構, 即: 模型與數據混合并行的增量學習方法. 所提方法由不同構建方式的左右兩個SCNs模型組成, 以快速準確地確定最佳隱含層節點, 其中左側采用點增量網絡(PSCN), 右側采用塊增量網絡(BSCN); 同時每個模型建立樣本數據的動態分塊方法, 從而加快候選“節點池”的建立、降低計算量. 所提方法首先通過大規?;鶞蕯祿M行了對比實驗, 然后應用在一個實際工業案例上, 表明其有效性.
                    • 圖  1  模型并行結構圖

                      Fig.  1  The structure diagram of model parallelism

                      圖  2  數據并行策略

                      Fig.  2  Strategy of data parallelism

                      圖  3  不同算法綜合性能比較

                      Fig.  3  Comparison of comprehensive performance of different algorithms

                      圖  4  模型的收斂曲線

                      Fig.  4  Convergence curve of HPSCNs

                      圖  5  模型的逼近特性

                      Fig.  5  Approximation performance of HPSCNs

                      表  1  基準數據集說明

                      Table  1  Specification of benchmark data sets

                      數據集屬性樣本數
                      輸入變量輸出變量
                      DB114424 160
                      DB212110 000
                      DB310140 768
                      DB426114 998
                      下載: 導出CSV

                      表  2  分塊數遞增區間長度及其上下界

                      Table  2  Incremental interval length of block number and its upper and lower bounds

                      $L_{en}^k$$L_{\max }^k$$L_{\min }^k$
                      50500
                      10015050
                      150300150
                      ·········
                      下載: 導出CSV

                      表  3  不同算法性能比較

                      Table  3  Performance comparison of different algorithms

                      數據集算法t(s)kL
                      DB1SC-III24.35$\pm $1.69164.40$\pm $7.76164.40$\pm $7.76
                      ${\rm{BSC - }}{{\rm{I}}_3}$12.60$\pm $1.2169.20$\pm $3.03207.60$\pm $9.09
                      ${\rm{BSC - }}{{\rm{I}}_5}$9.41$\pm $1.3344.00$\pm $3.24220.00$\pm $16.20
                      ${\rm{HPSCN}}_1^1$3.48$\pm $0.38122.40$\pm $8.02122.40$\pm $8.02
                      ${\rm{HPSCN}}_3^1$3.03$\pm $0.2863.40$\pm $4.16162.80$\pm $7.90
                      ${\rm{HPSCN}}_5^1$2.96$\pm $0.1945.00$\pm $2.83215.00$\pm $9.71
                      DB2SC-III26.97$\pm $2.54300.00$\pm $14.18300.00$\pm $14.18
                      ${\rm{BSC - }}{{\rm{I}}_3}$14.66$\pm $1.33120.40$\pm $3.98361.20$\pm $11.93
                      ${\rm{BSC - }}{{\rm{I}}_5}$11.01$\pm $1.0778.80$\pm $2.91394.00$\pm $14.87
                      ${\rm{HPSCN}}_1^1$7.22$\pm $0.95239.30$\pm $14.55239.3$\pm $14.55
                      ${\rm{HPSCN}}_3^1$5.47$\pm $0.33123.50$\pm $3.34301.90$\pm $10.99
                      ${\rm{HPSCN}}_5^1$4.39$\pm $0.4281.80$\pm $3.74378.60$\pm $16.54
                      DB3SC-III18.04$2.15106.60$\pm $3.36106.60$\pm $3.36
                      ${\rm{BSC - }}{{\rm{I}}_3}$8.96$\pm $1.2139.80$\pm $2.28119.40$\pm $6.84
                      ${\rm{BSC - }}{{\rm{I}}_5}$6.81$\pm $0.5525.20$\pm $1.10126.00$\pm $5.48
                      ${\rm{HPSCN}}_1^1$3.45$\pm $0.2497.00$\pm $2.6597.00$\pm $2.65
                      ${\rm{HPSCN}}_3^1$2.05$\pm $0.1341.20$\pm $2.17106.40$\pm $4.39
                      ${\rm{HPSCN}}_5^1$1.88$\pm $0.1225.00$\pm $1.22121.00$\pm $6.44
                      DB4SC-III9.16$\pm $0.34161.20$\pm $2.56161.20$\pm $2.56
                      ${\rm{BSC - }}{{\rm{I}}_3}$3.79$\pm $0.6854.20$\pm $0.84162.60$\pm $2.51
                      ${\rm{BSC - }}{{\rm{I}}_5}$2.59$\pm $0.1333.40$\pm $0.89167.00$\pm $4.47
                      ${\rm{HPSCN}}_1^1$4.23$\pm $0.13154.80$\pm $2.59154.80$\pm $2.59
                      ${\rm{HPSCN}}_3^1$2.01$\pm $0.1359.00$\pm $2.00162.60$\pm $2.41
                      ${\rm{HPSCN}}_5^1$1.36$\pm $0.1134.20$\pm $1.09166.20$\pm $3.03
                      下載: 導出CSV

                      表  4  不同塊寬的算法性能比較

                      Table  4  Performance comparison of algorithms with different block sizes

                      數據集算法nRnLEff (%)
                      DB1${\rm{HPSCN}}_1^1$61.361.149.9
                      ${\rm{HPSCN}}_2^1$63.822.426.0
                      ${\rm{HPSCN}}_3^1$52.812.619.3
                      ${\rm{HPSCN}}_5^1$42.52.55.6
                      ${\rm{HPSCN}}_{10}^1$24.20.62.4
                      DB2${\rm{HPSCN}}_1^1$119.2120.150.2
                      ${\rm{HPSCN}}_2^1$115.056.432.9
                      ${\rm{HPSCN}}_3^1$99.224.319.7
                      ${\rm{HPSCN}}_5^1$74.27.69.3
                      ${\rm{HPSCN}}_{10}^1$44.60.40.9
                      DB3${\rm{HPSCN}}_1^1$48.448.650.1
                      ${\rm{HPSCN}}_2^1$40.823.436.4
                      ${\rm{HPSCN}}_3^1$33.67.618.4
                      ${\rm{HPSCN}}_5^1$24.01.04.0
                      ${\rm{HPSCN}}_{10}^1$13.60.21.4
                      DB4${\rm{HPSCN}}_1^1$77.377.550.0
                      ${\rm{HPSCN}}_2^1$64.229.431.4
                      ${\rm{HPSCN}}_3^1$51.87.212.2
                      ${\rm{HPSCN}}_5^1$33.01.23.5
                      ${\rm{HPSCN}}_{10}^1$17.00.21.1
                      下載: 導出CSV
                      360彩票
                    • [1] 1 Chen Xing, Niu Ya-Wei, Wang Guang-Hui, Yan Gui-Ying. MKRMDA: Multiple Kernel Learning-based Kronecker Regularized Least Squares for MiRNA-disease Association Prediction. Journal of Translational Medicine, 2013, 15(1): 251?264
                      [2] 周平, 劉記平. 基于數據驅動多輸出ARMAX建模的高爐十字測溫中心溫度在線估計. 自動化學報, 2018, 44(3): 552?561

                      2 Zhou Ping, Liu Ji-ping. Data-driven Multi-output ARMAX Modeling for Online Estimation of Central Temperatures for Cross Temperature Measuring in Blast Furnace Ironmaking. Acta Automatica Sinica, 2018, 44(3): 552?561
                      [3] 湯健, 喬俊飛, 柴天佑, 劉卓, 吳志偉. 基于虛擬樣本生成技術的多組分機械信號建模. 自動化學報, 2018, 44(9): 1569?1589

                      3 Tang Jian, Qiao Jun-Fei, Chai Tian-You, Liu Zhuo, Wu ZhiWei. Modeling Multiple Components Mechanical Signals by Means of Virtual Sample Generation Technique. Acta Automatica Sinica, 2018, 44(9): 1569?1589
                      [4] Witten Ian H, Frank E, Hall M A. Data Mining: Practical Machine Learning Tools and Technique, third edition. Amsterdam: Morgan Kaufmann, 2011
                      [5] 5 Pao Y H, Takefji Y. Functional-link Net Computing: Theory, System Architecture, and Functionalities. Computer, 1992, 25(5): 76?79 doi: 10.1109/2.144401
                      [6] 6 Schmidt W F, Kraaijveld M A, Duin R P W. Feedforward neural networks with random weights. International Conference on Pattern Recognition IEEE Computer Society, 1992
                      [7] 7 Cao Wei-Peng, Wang Xi-Zhao, Ming Zhong, Gao Jin-Zhu. A Review on Neural Networks with Random Weights. Neurocomputing, 2018, 275: 278?287 doi: 10.1016/j.neucom.2017.08.040
                      [8] 8 Scardapane S, Wang Dian-Hui. Randomness in Neural Networks: An Overview. Wiley Interdisciplinary Reviews-Data Mining and Knowledge Discovery, 2017, 7(2): e1200 doi: 10.1002/widm.1200
                      [9] 9 Lu Jing, Zhao Jian-Wei, Cao Fei-Long. Extended Feed Forward Neural Networks with Random Weights for Face Recognition. Neurocomputing, 2014, 136: 96?102 doi: 10.1016/j.neucom.2014.01.022
                      [10] 10 Dai Wei, Liu Qiang, Chai Tian-You. Particle Size Estimate of Grinding Processes Using Random Vector Functional Link Networks with Improved Robustness. Neurocomputing, 2015, 169: 361?372 doi: 10.1016/j.neucom.2014.08.098
                      [11] 11 Dai Weo, Chen Qi-Xin, Chu Fei, Chai Tian-You. Robust Regularized Random Vector Functional Link Network and Its Industrial Application. IEEE Access, 2017, 5: 16162?16172 doi: 10.1109/ACCESS.2017.2737459
                      [12] 12 Li Ming, Wang Dian-Hui. Insights into Randomized Algorithms for Neural Networks: Practical Issues and Common Pitfalls. Information Sciences, 2017, 382: 170?178
                      [13] 13 Gorban A N, Tyukin I Y, Prokhorov D V, Sofeikov K I. Approximation with Random Bases: Pro et contra. Information Sciences, 2016, 364: 129?145
                      [14] 14 Wang Dian-Hui, Li Ming. Stochastic Configuration Networks: Fundamentals and Algorithms. IEEE Transactions on Cybernetics, 2017, 47(10): 3466?3479 doi: 10.1109/TCYB.2017.2734043
                      [15] 15 Zhu Xiao-Long, Feng Xiang-Chu, Wang Wei-Wei, Jia XiXi, He Rui-Qiang. A Further Study on the Inequality Constraints in Stochastic Configuration Networks. Information Sciences, 2019, 487: 77?83 doi: 10.1016/j.ins.2019.02.066
                      [16] 16 Wang Dian-Hui, Cui Cai-Hao. Stochastic Configuration Networks Ensemble for Large-Scale Data Analytics. biochemical treatment process. Information Sciences, 2017, 417: 55?71
                      [17] 17 Sheng Zhui-Yong, Zeng Zhi-Qiang, Qu Hong-Quan, Zhang Yuan. Optical fiber intrusion signal recognition method based on TSVD-SCN. Optical Fiber Technology, 2019, 48: 270?277 doi: 10.1016/j.yofte.2019.01.023
                      [18] 王前進, 楊春雨, 馬小平, 張春富, 彭思敏. 基于隨機配置網絡的井下供給風量建模. 自動化學報, doi: 10.16383/j.aas.c190602

                      18 Wang Qian-Jin, Yang Chun-Yu, Ma Xiao-Ping, Zhang Chun-Fu, Peng Si-Min. Underground Airflow Quantity Modeling Based on SCN. Acta Automatica Sinica, $ref.ref_year doi: 10.16383/j.aas.c190602
                      [19] 19 Wang Dian-Hui, Li Ming. Deep Stochastic Configuration Networks with Universal Approximation Property. In 2018 International Joint Conference on Neural Networks, 2018
                      [20] 20 Wang Dian-Hui, Li Ming. Robust Stochastic Configuration Networks with Kernel Density Estimation for Uncertain Data Regression. Information Sciences, 2017, 412-413: 210?222 doi: 10.1016/j.ins.2017.05.047
                      [21] 21 Li Ming, Huang Chang-Qin, Wang Dian-Hui. Robust Stochastic Configuration Networks with Maximum Correntropy Criterion for Uncertain Data Regression. Information Sciences, 2019, 473: 73?86 doi: 10.1016/j.ins.2018.09.026
                      [22] 22 He Qing, Shang Tang-Feng, Zhuang Fu-Zhen, Shi Zhon-Zhi. Parallel Extreme Learning Machine for Regression Based on MapReduce. Neurocomputing, 2013, 102: 52?58 doi: 10.1016/j.neucom.2012.01.040
                      [23] 23 Duan Ming-Xing, Li Ken-Li, Liao Xiang-Ke, Li Ke-Qin. A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning Machine. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2337?2351 doi: 10.1109/TNNLS.2017.2654357
                      [24] 24 Wang Yue-Qing, Dou Yong, Liu Xin-Wang, Lei Yuan-Wu. PR-ELM: Parallel Regularized Extreme Learning Machine Based on Cluster. Neurocomputing, 2016, 173: 1073?1081 doi: 10.1016/j.neucom.2015.08.066
                      [25] Lancaster P, Tismenetsky M. The Theory of Matrices: With Applications. Elsevier, 1985
                      [26] 26 Dai Wei, Li De-Peng, Zhou Ping, Chai Tian-You. Stochastic Configuration Networks with Block Increments for Data Modeling in Process Industries. Information Sciences, 2019, 484: 367?386 doi: 10.1016/j.ins.2019.01.062
                      [27] 27 Dai Wei, Zhou Ping, Zhao Da-Yong, Lu Shao-Wen, Chai Tian-You. Hardware-in-the-loop Simulation Platform for Supervisory Control of Mineral Grinding Process. Powder technology, 2016, 288: 422?434 doi: 10.1016/j.powtec.2015.11.032
                      [28] UCI Machine Learning Repository. [Online], available: https://archive.ics.uci.edu, 2013
                      [29] KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework. [Online], available: http://www.keel.es/, 2011
                    • 加載中
                    計量
                    • 文章訪問數:  3511
                    • HTML全文瀏覽量:  917
                    • 被引次數: 0
                    出版歷程
                    • 收稿日期:  2019-05-27
                    • 錄用日期:  2019-12-02
                    • 網絡出版日期:  2019-12-24

                    目錄

                      /

                      返回文章
                      返回