An email I got with all the wrong reasons to publish a paper

This didn’t happen to a fried (or a friend of a friend…). This happened to me. I’m just removing personal details of the person that sent the email (“hate the deed, not the doer” or whatever works as the English translation of the Spanish saying “se dice el pecado pero no el pecador”).

Some time ago I was invited to co-author a research paper based on my expertise on ATL (the paper was about transforming from one modeling language to UML). I didn’t know the researcher that invited me but a quick look at his DBLP showed a respectable publication record and there was already another co-author that I did know and that was well-known in the community. So at that point I just asked him to send me a draft of the paper to get a better idea of what they were trying to do and evaluate my possible contribution.

Once I read the draft I got back expressing two serious concerns about the paper. In short:

  1. I couldn’t see why amybody would need that transformation.
  2. The transformation itself was straightforward (basically everything was a 1 to 1 mapping) so by itself it wasn’t a contribution either

This is the answer I got (minimally edited to preserve anonymity of all people involved). Judge by yourself what this says about how some researchers behave (and, to be clear, I don’t want to put all the blame on that specific person, I do believe that this publish or perish philosophy imposed by many evaluation agencies is to blame as well.

First regarding my concerns:

Yes, the case studies aren’t real. I have talked to Dr. X and Dr. Y who have great expertise in the area and I found out that Z is not really widespread in industry and therefore industrial case studies are almost impossible to come by. Unfortunately, this piece of information can a little too late.
Yes, the transformation is too straightforward. Not only that, but we are actually abstracting the model because Z are semantically richer UML activity diagrams.

And his reaction to them:

Our aspirations need to curbed. This will not turn into an IEEE TSE, TOSEm or even IST paper. It may fly in other lower level journals. I am thinking JSS or Sosym (or even lower), solely for the fact that these journals advocate modeling for the sake of modeling (especially Sosym which is purely a modeling journal).

So basically, my concerns are true but they are not a problem per se but only a problem in the sense that may prevent the paper to be published in top journals. And now his proposal

Given the drawbacks of the paper, a great deal of thinking needs to be put in deciding which journal to submit too. The choice of the journal will greatly affect the chances of success. The strategy is to beef up as many sections of the paper as possible. The goal here is to blow away the reviewers via complexity. This may work for Sosym because the reviewers will not care much for the motivation nor will they dwell over the non-real-world case studies. I have published 3 papers in Sosym that had a similar situation, including two papers with validation sections comprising of only exemplars. We can also use the complexity to target softer journals (i.e. information systems journals). IS journal reviewers I am sure will be taken by the complexity but they may be smart enough to see a lack of motivation.

In short, let’s try to fool the reviewers in believing we have actually something in our hands!. Love his consideration for IS reviewers that could be smart enough to caught our lies! Needless to say I kindly declined to participate in the paper but I can’t help but feeling angry with the situation. These are tough times to get research funding and this kind of behaviour is screwing us up even more.

Anyone with similar personal experiences?

Posted in funny, publishing | Leave a comment

Where am I supposed to submit complete research works?

The other day we tried to submit a journal paper summarizing the PhD work of one of my students. He had published a couple of conference papers and one workshop paper covering specific parts of his work and, after the PhD, he wanted to write a journal paper presenting the complete method. This is, I would say, a typical publication pattern.

The problem came when we tried to submit this paper to a journal. The paper was long, to be precise, 16.000 words long. This didn´t seem like a problem to me. This was the complete overview of 3-year work period and, IMHO, deserved this space in order to provide enough details of all the components and the relationships between them (remember that a typical 15-page LNCS paper is around 5.000 words so we’re talking about a paper just triple that size).

But, as usual, I was wrong. Journals do not seem interested in publishing high quality works regardless their size (not saying that mine was a high-quality work, but they don´t know either, they didn´t take the time to check). Journals just want to publish the more papers the better. The editor-in-chief immediately replied

The paper comes across as very long. This means that it will be difficult to find reviewers for the paper. Furthermore, the contribution of the paper will be judged in relation to its length. I would strongly suggest that you try to shorten the paper.

In a posterior email, the editor clarified that by shorten he meant no more than 11.000 words so basically he was asking to remove one third of the paper.

And it turns out this is a common requirement. The ACM TOSEM journal says:

Extremely long submissions — as a general rule, those that exceed approximately 11,000 words — may be returned without review at the discretion of the editor-in-chief. If placed into the review process, such submissions are not guaranteed review or publication in a timely fashion.

Since when a 11.000 words paper is an extremely long submission? Researchers always say that it´s a bad practice to publish only small increments over previously published works but journals are forcing us to do exactly that and stick to the minimum 30% novelty rule. Specially now that conference papers are following the complete opposite path and get larger and larger (many conferences now accept papers up to 18-20 pages in LNCS format).

And yes, there is an answer to my initial question. I could publish the complete research work in an open repository like arXiv but, unfortunately, that’s not a valid option for my student.

(btw, if you wondered what happened with the paper, we managed to reduce it to 12.500 words, it’s now a worse paper but hopefully still good enough to be published, and at least this time the editor sent it out for review).

Posted in publishing | Leave a comment

Scientists’ (sad) behaviour as seen by Team Geek book

I’m really enjoying the book Team Geek: A Software Developer’s Guide to Working Well with Others and I strongly recommend it to any software developer out there but that´s not why I’m mentioning here.

I’m doing it because it includes a comparison between professional scientists and software developers as a way to convince software developers not to work alone and instead join the open source movement:

Professional science is supposed to be about the free and open exchange of information. But the desperate need to “publish or perish” and to compete for grants has had exactly the opposite effect. Great thinkkers don´t share ideas. They cling to them obsessively, do their research in private, hide all mistakes a long the path and then ultimately publish a paper making it osund like the whole process was effortless and obvious. And the results are often disastrous: they accidentally duplicated someone else´s work or made an undetected mistake early on … The amount of wasted time and effort is tragic

(and of wasted public money I´d add)

True, nothing really new here (I already touched this same topic in the post: “Be honest, curing cancer is not your primary goal“) but it surprised me that the same perception was shared by people outside our community. I’d say this is a good thing, the more pressure we have to change the way research is done, the better.

Posted in doing research, publishing | Leave a comment

What stats would you like to know for every conference?

In the opening session of any conference, the PC Chairs give a brief presentation of the conference. This typically includes informing about the number of abstracts and full papers submitted, the number of papers accepted, the corresponding acceptance rate and some kind of map / graphic displaying the same information by countries.

Usually, that’s it. For the opening session of ECMFA’14 we wanted to give some more (hopefully interesting) data. In the end, due to time constraints, what we gave as additional data was:

  1. Percentage of accepted papers where none of the authors was (or had been in the last four years) a PC member of the conference
  2. Acceptance rates of papers were at least one author was (or had been) a PC member
  3. Number of papers where none of the authors had participated in the community (as a PC member or author) before (again “before” means in the last 4 years)

With the first two we wanted to show that you didn’t need to be a PC member to get your paper in the conference (for ECMFA 42% of papers were from non-PC members) and that having a PC member as co-author did not increase dramatically the probability of getting your paper accepted (acceptance rate for PC co-authored papers was only 10% higher than acceptance rates for papers with no PC member). The third was a way to see how endogamic was the conference (turned out to be quite a lot since only one paper had a complete set of “fresh” authors).

Can I ask you if you would like all conferences to include these three stats in their presentation? And, regardless of your answer, what other data/statistic would you like to know about a confenrence?

Posted in organization, publishing | 15 Comments

Real Workshops do NOT publish papers

On Tuesday, I attended this workshop and I really enjoyed it a lot. The difference with other workshops? This workshop was by invitation (open to all the particpiants in the ASE PC Board meeting taking place the day after) so no call for papers, no publications of any kinds of proceedings, no restriction on the topics to talk about, no nothing.

For me, the key aspect was the fact that authors were not there to talk about any specific paper. In general, most workshops work by publishing a call for papers and you must submit a paper to the workshop in order to be able to do a presentation in the workshop. Too often, this results in a workshop full of delta papers (i.e. papers that are just a minor improvement wrt previous papers) quickly written to justify the attendance to the workshop. The problem is that, then, during the workshop, authors feel obliged to talk about that specific paper instead of taking the opportunity to have a more open discussion which results in boring and uninteresting presentations.

I’d like to see more workshops going back to their original mission: be a place for discussion and exchange of ideas, instead of becoming mini conferences!

Posted in organization, presenting | 2 Comments

Should you submit to your own workshop? My pragmatic response

A typical question in every workshop I co-organize is whether we (the organizers) are allowed to submit papers to the workshop.

I almost never do it but I have no problem with other co-organizers doing it (I’m assuming we are talking about real workshops, not about conferences disguised as workshops) with just one pragmatic condition: I don’t want this to add any extra work to my organization duties. By this I mean that I’m not going to manage papers from organizers outside EasyChair (or any other conference management system) to ensure the anonymity of the reviewers. This is not only a pain in the ass for me but also for the affected reviewers and the whole PC.

So, do you want to submit a paper to the workshop you’re organizing? Fine with me but then I´ll just trust that you’ll look the other way when your reviews start coming.

Posted in publishing | 2 Comments

Great site about spurious correlations

I’m sure any researcher is perfectly aware of the differences between Correlation and Causality (if not, read this and you´ll understand why banning Internet Explorer is not likely to stop murders in the US).

But the fact that we do know this doesn´t mean we are not tempted to forget it (causal relationships look great in papers!). In those situations, please take a look at the spurious correlations site, full of amazing data correlations (e.g. Nicholas cage appearances in films and people drowning or consumption of mozzarella cheese and number of civil engineering doctorates) for a a reality check!.

Now, jokes apart, make sure young students/researchers take a look at this site and we may avoid quite a few rejected papers in the future.

Posted in doing research, funny | 1 Comment