Current Status and Challenges of Smart Grid Big Data Processing Technology Song Yaqi, Zhou Guoliang, Zhu Yongli 2 (1. School of Control and Computer, North China Electric Power University, Baoding 071003, China; 2 National Key of New Energy Power System. It should be pointed out that given the current cloud platform The real-time performance of receiving smart grid monitoring data cannot be guaranteed. Several front-end machines can be set in front of data access and information integration, which is responsible for receiving alarm information or monitoring data sent from the communication network in real time, and is responsible for when the cloud platform cannot respond. Temporary storage.
Smart grid various applications production control system power management and management power camp state inspection risk assessment system measurement system evaluation system task management, scheduling and monitoring Hadoop cloud computing system parallel data warehouse real-time database data access and information integration smart grid big data multi-level Storage Systems In addition, the data format in the smart grid is very different from traditional business data and has its own characteristics. For example, in fault recording and status monitoring of power transmission and transformation equipment, waveform data is more, and waveform data is essentially different from traditional commercial data, and has the characteristics of fast data generation. Therefore, it is necessary to study the format of the big data storage for the smart grid, which is conducive to subsequent data analysis and calculation.
In the smart grid environment, various types of data are heterogeneous and cannot be described by existing simple data structures. Computer algorithms are relatively inefficient in processing complex structural data, but processing homogeneous data is very efficient. Therefore, how to organize data into a reasonable homogeneous structure is an important issue in big data storage processing. In addition, there is a large amount of unstructured and semi-structured data in the smart grid, and how to transform this data into a structured format is a major challenge.
3.2 Real-time data processing technology 3.2.1 Timeliness of data processing For big data, data processing speed is very important. In general, the larger the data size, the longer the analysis processing will take.
The traditional data storage scheme is designed for a certain amount of data, and the processing speed may be very fast within its design range, but it cannot meet the requirements of big data. In the future smart grid environment, real-time data processing is required from the power generation, power transmission and transformation links to the power consumption. Current cloud computing systems can provide fast service, but may suffer from short-term network congestion, or even a single server failure, and cannot guarantee response time.
Memory-based databases are getting more and more attention. An in-memory database is a database that operates directly on data in memory. Compared to disk, the data read and write speed of memory is several orders of magnitude higher. Saving data in memory is much better than accessing from disk. The power system has begun to use in-memory databases to improve real-time performance. For example, in view of the power shortage in some parts of China last year and the overcapacity in another part of the country, SAP introduced a smart meter analysis solution based on the HANA in-memory database, hoping to link the data involved in the smart grid with the data of large power users. Integrate and integrate analysis to achieve analysis of power consumption in various places to take appropriate preventive measures.
Searching for keywords in big data sets is also an important challenge. It is obviously not feasible to find a record that meets the requirements by scanning the entire data set, even if it is speeded up by parallel processing techniques like MapReduce. Helping the lookup by pre-indexing the data is a quick and conducive way to save system resources. At present, the design of general index structure only supports some simple data types, and big data requires the establishment of a suitable index structure for complex structure data, which is also a huge challenge. For example, the multi-dimensional data collected by the Internet of Things has a growing amount of data, and there are requirements for query time limits. It is necessary to continuously update the index structure, and the design of the index is very challenging. The following are the challenges of data processing in smart grid big data from power generation, power transmission and power consumption.
3.2.2 Power Generation The characteristics of the power generation enterprise are continuous production and high degree of automation. It requires real-time monitoring of the entire process, high-speed real-time data processing, long-term historical data storage, and integration and sharing of production information. Studies have shown that a properly operating SCADA system that receives monitoring data delays of more than 50 ms can lead to erroneous control strategies; and studies have shown that SCADA systems fail when using the most common TCP/IP protocol in the Internet environment. The main reason is that the TCP protocol is performing flow control and data error correction, resulting in data delay. Future smart grid solutions will require real-time response, even in the event of a node failure. Current relational database systems and cloud computing systems are designed to handle permanent, stable data. The relational database emphasizes the integrity and consistency of the maintenance data; the cloud computing system emphasizes reliability and scalability, but it is difficult to take into account the timing constraints of the data and its processing, and cannot meet the needs of real-time application of industrial production management.
3.2.3 Status monitoring of power transmission and transformation links has high requirements on the performance or real-time performance of data storage and processing platforms. Although cloud computing technology can effectively process big data, it needs to further enhance the storage of massive monitoring data by cloud platforms. Take performance to meet real-time requirements. In the past, large-scale power outages were initially caused by environmental factors, such as line trips caused by strong winds. The monitoring scope of the existing SCADA system is limited to the main parameters of the system, and the lack of information on the health status of the important equipments constituting the system makes it difficult for the operating personnel to correctly handle the accident. In the future, smart grids require fault self-healing functions. The SCADA system must have monitoring data of the entire network, and the state data of the power equipment needs to be included. This puts higher requirements on the real-time processing of the platform.
The instability of the new green energy power generation causes fluctuations in the power grid, which creates great pressure on the entire power grid dispatch. At present, the grid dispatching and control model cannot handle the fluctuations and unpredictable behavior of such a large number of small power generation systems. The latest research shows that in order to support this situation, it is necessary to create a new type of grid condition monitoring system that can track the real-time status of the grid more finely. Therefore, future SCADA systems need to process several orders of magnitude more monitoring data in real time.
3.2.4 Power Consumption In the future smart grid environment, the home may be equipped with a variety of electrical energy and power monitoring devices to achieve low-cost electricity consumption and match the load of the power grid. For example, an electric water heater may choose to operate at a low power consumption during the night; the air conditioner will automatically adjust in real time according to parameters such as user comfort, electricity price, and grid load. To some extent, we can think that the SCADA system has entered the ordinary family, and the real-time data processing of the electricity link has become more and more important.
3.3 Heterogeneous multi-data source processing technology 3.3.1 Integration of heterogeneous information In the future, smart grids require multiple links such as power generation, transmission, substation, power distribution, power consumption, and scheduling to achieve comprehensive information collection, smooth transmission, and high efficiency. Processing, supporting the high integration of power flow, information flow, and business flow. Therefore, the primary function is to realize the integration of large-scale multi-source heterogeneous information, and provide a data center with resource intensive configuration for the smart grid. For the massive heterogeneous data, how to construct a model to regulate it, how to achieve data fusion based on the model, and to effectively store and efficiently query it is an urgent problem to be solved.
Most of the information systems of the power grid are based on the needs of the business or the department. Different platforms, application systems and data formats exist, resulting in the dispersion of information and resources. The heterogeneity is serious and cannot be shared horizontally. It is difficult to communicate vertically between the upper and lower levels, for example: There are various information systems in the power system such as monitoring, energy management, power distribution management, and market operation. Most of them are independent of each other, and data information cannot be shared. The integration of independent systems using the cloud platform enables information interoperability between these distributed and isolated systems.
In addition, the smart grid's infrastructure is large, large and distributed across different locations. For example, the information platform of the State Grid Corporation establishes a two-level data center at the company headquarters and various network companies to realize the three-tier application of the company headquarters, network province companies, and prefectures and cities. How to effectively manage these infrastructures and reduce the operating costs of data centers is a huge challenge.
3.3.2 Efficient management of various types of grid data In the heterogeneous multi-source information fusion and management of smart grids, it is necessary to establish the information interoperability model of 61970. Since the data types in the smart grid are more than the types involved in IEC61850, the application of multi-layer knowledge structure and semantic methods, the establishment of domain-oriented analysis models and semantic-based service models is an optional method. Based on the theory of statistical learning, support vector machine, correlation vector machine and association rule mining, this paper studies the integration scheme and real-time mining algorithm of heterogeneous data fusion and mining. Since the deterioration of the state of the equipment is a process from quantitative change to qualitative change, the mining of time series data such as oil chromatography accumulated over many years is more meaningful. At present, there are some research results of such big data mining, but the degree of practicality is not high.
3.4 Big Data Visualization Analysis Technology Faced with massive amounts of smart grid data, how to present it to users in an intuitive and easy-to-understand way with limited screen space is a very challenging task. Visualization methods have proven to be an effective method for solving large-scale data analysis and are widely used in practice. Large-scale data sets generated by various applications in the smart grid, including high-precision, high-resolution data, time-varying data, and multivariate data. A typical data set can reach a terabyte set. How to extract useful information quickly and efficiently from these huge and complex data becomes a key technical difficulty in smart grid applications. Visualization draws data into high-precision, high-resolution images through a series of sophisticated algorithms, and provides interactive tools that make effective use of human visual systems and allow real-time changes to data processing and algorithm parameters to observe, characterize, and quantify data. analysis.
The challenges in this area mainly include the scalability of visualization algorithms, parallel image synthesis algorithms, extraction and display of important information.
4 Conclusion The future smart grid will be a panoramic real-time grid relying on big data processing and analysis technology. Cloud computing provides a platform for storage and analysis of this heterogeneous and diverse data. After the platform runs for a period of time, big data will inevitably be generated. The cloud platform and big data analysis will provide support for state maintenance of power equipment, self-healing of power grids, and interoperability of isolated information systems, and become an important candidate for low-cost, good systems. Extensive (unlimited storage capacity), high reliability, parallel analysis, etc. There are several systems in the world that have been put into practical operation, but there are still many challenges in real-time, data consistency, privacy and security. Need to find the corresponding solution. Big data processing technology is still lacking, and people have to explore.
Socket Weld Flange is flange type of ANSI B16.5 norm have class rating 150 300 600 900 1500 2500 facing type raised face RF and ring type joint RTJ, flange size NPS 1/2"-24", steel ASTM A105 A350LF A181 GR60 and more, socket welding flanges are similar to Slip On Hubbed flanges but difference for the flange center hole, there is a step in the flange center hole for socket weld flange.
Socket Weld Flange
Socket Weld Flange,Socket Weld Flange Ansi,Socket Weld Flanges Rtj,Socket Welding Flange
Jinan Hyupshin Flanges Co., Ltd , https://www.iflange.com