kumoh national institute of technology
Networked Systems Lab.

Review Comment

NSL > Education> Review Comment
Alif - ETFA 2017
By :
Date : 2017-06-19
Views : 396

Hello everyone, please kindly the review comments from my paper in ETFA. Learn and be better for future submission.

Thank you.


Dear Alif Akbar Pranata,

We regret to inform you that your paper A Robust Openflow Controller for Software-defined Real-time Ethernet has been REJECTED for presentation at the ETFA 2017 - IEEE International Conference on Emerging Technology & Factory Automation, to be held in Limassol, Cyprus, from 13th to 15th September 2017.

The selection process was competitive, and due to time and space limitations, we could only choose a limited number of the submitted papers to appear in the program. The selection decisions are the result of a thorough reviewing process. The reviewers have stated, often in detail, which deficiencies they see in the submissions. You can access the reviewer comments for your paper by logging into the submission system using the access code you received when submitting the paper. We hope you will find that the reviews provide useful feedback for your future submissions.

We hope that you will consider a participation in the ETFA2017 conference, and look forward to meeting you in Limassol.

Sincerely yours,
Charis Theocharides and Lukasz Wisniewski
ETFA 2017 Work in Progress Co-Chairs

----------------------- REVIEW 1 ---------------------
PAPER: 260
TITLE: A Robust Openflow Controller for Software-defined Real-time Ethernet
AUTHORS: Alif Akbar Pranata, Tae Soo Jun and Dong-Seong Kim

Overall evaluation: -2 (reject)

----------- Overall evaluation -----------
The paper presents requirements for building a robust OpenFlow controller for Software-defined real-time Ethernet. The authors provide four valid requirements for SDNs in RTEs and aim to solve the first requirement with an implementation and test, and propose a potential solution for the second requirement. But the presented solution strategies as well as their implementation and test are weak.

The first requirement is implemented as a "delay module" that aims to guarantee low delay and zero packet loss in a network. For this they "only consider the first packet of a flow to determine the performance of delay module", i.e. they only send the first packet of a flow to the controller, evaluate it, and install all required flows rules at the switches that will be used to forward following packets. With this, they aim to reduce the latency of packet transmission by minimizing the controller interaction and its workload. It is unclear how this strategy could lead to a guaranteed zero packet loss especially because the authors don't describe a scenario where such packet loss could occur. The tests that were executed don't provide more information on this. They claim that their strategy leads to minimized delay and zero packet losses, and they also compare their idea to tests that were executed in a cited paper. But since details on the executed tests are not given by the!
author, the results are not comparable and lack significance. Furthermore, the presented strategy can be seen as standard behavior of an SDN (It is also unclear how this strategy would even help to reduce packet loss in e.g. a situation where a link fails and the packet transmission must be switched to an alternative route. Also the packet delay is not only affected by the controller interaction but mainly by the utilization of the actual communication links). A valuable extension would have been to investigate the delay-influence of even this single controller interaction compared to a non-SDN or to handle the limited capacity of flow tables in switches that can probably not store flows that are created for each individual interaction between two entities.

The described scheduling module is more interesting and provides a time-triggered SDN flow approach. However, this is also weak since the implementation of such will most probably need special SDN switches that are aware of the different time slots and know when which flow is active. Details on a potential implementation, the expected problems and solution strategies, and especially on use cases and comparisons to existing time-triggered Ethernet protocols are missing at all and are referred to as future work. It would also be interesting to review how this solution compares e.g. to current standardization efforts in TSN and also a combination of the two domains (SDN and TSN).

Due to this, the paper should not be accepted for presentation on the conference in this form. The implementation and test of the delay module is not very strong and shows large gaps. Furthermore the context of SDN, RTE, and its interconnection is not described well and also the motivation is unclear.

Finally, the text should be grammatically revised (e.g. use OpenFlow instead of Openflow; use Ethernet instead of ethernet). Also, there are some errors in the formulas and in TABLE 1 ("Set of time-triggered flows" is named FS instead of TS). Additionally, there is an error in the author name of [1]. The author is called "Schörghofer" and not "Schrghofer"

----------------------- REVIEW 2 ---------------------
PAPER: 260
TITLE: A Robust Openflow Controller for Software-defined Real-time Ethernet
AUTHORS: Alif Akbar Pranata, Tae Soo Jun and Dong-Seong Kim

Overall evaluation: 1 (weak accept)

----------- Overall evaluation -----------
General comments

- From my point of view the paper is very interesting and relevant for the audience, since it describes and ongoing proposal for developing an Openflow SDN controller suitable for Robust Real Time Ethernet Industrial Networks. The paper proposes what are the requirements such a controller should fulfill and focuses on two of them: (1) guarateeing zero packet losses and (2) deadline fulfillment of Real Time Traffic. The paper proposes an algorithm for the controller to achieve (1) and provides some simulation results that seem to be promising. Also, the paper proposes an schedule schema for (2), even though it does not present any simulation or real test that demonstrates its Real Time guarantees.

- Even though the paper is interesting, it has some important shortcomings regarding clearness and lack of justification for some important claims.

- In particular the tests and results presented to demonstrate that the Authors proposal fulfills (1) are controversial. Similarly, no result or formal demonstration/argumentation is provided to support that Authors proposal is going to fulfill (2). Therefore, in my opinion, Authors need to explicitly clarify that the results presented in this paper are preliminar.

- Please, for more details on these shortcomings refer to the specific comments below.

Specific comments


- What do authors specifically refer to as RTE networks? There are several proposals to make Ethernet adequate for real time industrial systems. Thus, it is necessary to clarify what specific RTE protocol or protocols are authors talking about and comparing their proposal to.

- Authors could provide some references about Openflow and further explain why it is more advantageous that other types of SDN controllers. In particular, why is Openflow more adequate for RTE industrial communications than other types of SDN controllers?


- It seems there is a contradiction between what is said at the beginning:

Application of SDN has been investigated definitively for industrial network application, as of by Henneke et al. [2] and Ka ́lma ́n [3]. (..) Du and Herlich [4] proposed an idea of integrating SDN for RTE. (..) Later in their next paper [1], they proved the concept by providing implementation setup on several network scenarios based on the requirement of SDN-enabled RTE.

and what is stated at the end of this Section:

All cited paper above works independently from those mentioned ideas (SDN for industrial network, software-defined RTE, and Openflow controller performance). (..)

Please, may authors revise this last statement and rewrite it so as to prevent such a contraction? Otherwise, it is not possible to fully understand what is the advantage of the authors proposal compared to previous ones.

- 2nd paragraph, 2nd sentence. This sentence is hard to follow. May I ask authors to revise it to make it fully understandable?

- Last sentence: (..) the main contribution of this paper is to introduce a scheme for robust Openflow controller to be applied in software-defined RTE by SIMULATING the network on a VIRTUAL environment of RTE. Until this point it seemed that authors proposed a first implementation based on a real, not virtual, RTE environment. Please, may I ask authors to make it clear from the very beginning (in the Abstract and Introduction) the kind of implementation/test they have conducted so far?


- Please, may you consider the following comments of the list of requirements that are specified for the enhanced Openflow controller.

- 1st requirement. There is an important typo here: non-deterministic should be deterministic.

- 2nd requirement. The text is hard to follow/understand. What do authors want to actually state here?

- 3rd requirement. It presents admission control as a mechanism to make the network secure, by identifying authorized and unauthorized packet flow. However, to my best knowledge, admission control rather refers to a set of mechanisms to provide the capacity of accepting/denying online changes in the traffic scheduling; based on either a given online schedulability analysis and/or an online policy to restrict who is authorized to request changes. Please, may I ask authors to consider also this alternative meaning of admission control or, at least, to mention and discuss a little bit about the necessity of admission control when it is understood from this alternative point of view?

- 4th requirement. It refers to the necessity of recovery and retransmission mechanisms. However, it is not clear whether authors are interested in robustness, reliability, availability, etc. Please, may I ask authors to clarify what is/are the specific attribute/s of dependability they are interested in?

- Last paragraph, 1st sentence just after the list of requirements. Authors claim: This paper enhances THE existing Openflow controller. To which specific realization/implementation of already existing Oplenflow controllers are they comparing their proposal?

- Last paragraph, 3rd sentence. Authors say: Hereafter, THE TWO PROPOSED requirements are referred to module name of delay module and scheduling module, and the definition of delay, latency, and jitter is considered equally as end-to-end delay transmission. However, it is not clear to WHICH TWO proposed requirements are they talking about? Note that this ambiguity is also present in the first sentence of Section IV. Please, may I ask to clarify this ambiguity in both Sections?


- 1st paragraph. Authors say: This paper mainly considers the first packet of a flow to determine the performance of delay module. Why it is enough to consider just the 1st packet of a given flow? Moreover, why is the flow authors are going to consider relevant/representative for/of the delay measurement?

A. Lower Delay and Zero Packet Losses Assurance

- 1st paragraph. Authors explain that their proposal to guarantee zero packet losses is to minimize the controller workload so as to increase the chances or packets being successfully transmitted. This is a controversial claim. From my point of view, minimizing packet losses is not equivalent to guaranteeing zero packet losses. Therefore I would suggest authors to clarify this issue from the very beginning, since one of the contributions of the paper is claimed to be that NO packet will be lost!

- Algorithm specified by pseudocode and its explanation. Both, the pseudocode and its explanation can be improved in terms of clarity; some parts of the explanation are hard to follow.

B. Transmission Process Scheduling

- Table 1. This table and the part of the main text that references this table uses two different acronyms to referring to the set of time-triggered flows: FS and TS.

- 2nd paragraph, 2nd sentence. This sentence is hard to follow and may be sintactically incorrect.

- List of constraints, 2nd item, expressions. These expressions and its explanation should be improved. It is necessary to clarify what does it mean that The source host has maximum one outgoing links and no incoming links and that the destination host has maximum one incoming links with no outgoing links. Do they mean that a host can only act as a sender or only as a receiver, but cannot act as sender and receiver? Moreover, on the one hand, please define what in(src(i)), out(dst(i)) mean. On the other hand, there is a syntactic error in the second term: V|src(i),dst(i)}.

- Last paragraph. Here it is claimed that the formulations given in this subsubsection maximizes the number of flows that are schedulable from a given set of flows. Nevertheless, this claim needs to be substantiated by means of further discussion. Otherwise, it is controversial; authors should demonstrate this claim.


- 1st paragraph. It eventually states: Moreover, installing some proactive flow entries based on other real-time requirements mentioned in the proof- of-concept of software-defined RTE by Herlich et al. [1] can help to MAXIMIZE THE RESULT. To WHICH RESULT does this statement refer to? What does it mean to MAXIMIZE?

- The section does not make it clear neither what is the topology of the network nor the conditions of the setup used by other authors to evaluate the number of packet losses with the software-defined RTE by Herlich et al. and RSTP (reference [1]). Were these two alternatives evaluated in [1] using the same topology and traffic as for the authors proposal?

The same doubts arise concerning the evaluation of the controller overhead carried out for JumpFlow in [5]. Certainly, authors explain that JumpFlow was not designed for industrial networks and RTE, but then, what is the point of comparing their proposal with JumpFlow?

- Figure 1. Please, describe a bit further what the blue lines mean in this figure. In particular, what is the percentage of each measured number of packet losses for RTSP?

- Last but one paragraph. It is not clear why or how the proposed scheduling schema does suit for DSN RTE. Authors explain the basic ideas of this schema, but they do not demonstrate its suitability.

----------------------- REVIEW 3 ---------------------
PAPER: 260
TITLE: A Robust Openflow Controller for Software-defined Real-time Ethernet
AUTHORS: Alif Akbar Pranata, Tae Soo Jun and Dong-Seong Kim

Overall evaluation: -1 (weak reject)

----------- Overall evaluation -----------
A robust flow controller for software-defined ethernet is described. Related work is stated.

Validation is done with powerlink, which is specially deigned to avoid collissions and with this paket loss.

The implementation of the Openflow does avoid paket loss. So I do not see the relation with the RTE example taken for the validation.

I do not see how this approach will allow RTE to be implemented over SDN.

----------------------- REVIEW 4 ---------------------
PAPER: 260
TITLE: A Robust Openflow Controller for Software-defined Real-time Ethernet
AUTHORS: Alif Akbar Pranata, Tae Soo Jun and Dong-Seong Kim

Overall evaluation: -1 (weak reject)

----------- Overall evaluation -----------
This paper presents a preliminary investigation on using and extending Openflow protocol for its effective use in real-time Ethernet ontext.
After some general considerations about low latency, zero packet loss, scheduling for deadline guarantee, admission control fault recovery and packet retransmission, the authors presents two contributions:
- an algorithm for guaranteeing the low delay and zero packet loss;
- formulated the scheduling of flows into a given set of links and time slots as an optimization problem.
Some very preliminary results are presented.
The low delay and zero packet loss guarantee algorithm is implemented in a so-called delay module (only first packet of a flow provoke a PACKET_IN which needs to be analysed by the SDN controller) and compared with the native openflow (every packet provokes a PACKET_IN). Obviously it results in the expected result of Figure 2 (no need to do any implementation of measurement to get the same conclusion!).
The scheduling optimization seems do not consider the multicast cases. In addition it is not applied to a numerical case.

The paper deals with a very interesting and important topic. However, the contribution is still too small, even for a WiP. So I think it is better to get more convincing results that will allow to producing a much better paper on such an important topic.