IEEE Bibliographies with Pandoc

I’ve recently rolled my own python pandoc-filter to parse the bibliography to be compatible with the IEEEtran.cls for IEEE style transactions for LaTeX submissions.
I thought I’d write it up here rather than forget the result — it took me all day to write only 30+ lines of filter code and most of that was figuring out how to debug a filter.

First off, if you’re just looking for the filter it can be found here. Ok! now let’s get into it…

The Problem




The problem begins with IEEEtran’s style guide only correctly formatting `proper` bibtex sections in the bibliography.
For instance:
\bibitem{b1}\hypertarget{ref-declerck2016cori}{} T. Declerck \emph{et al.}, ``Cori - a system to support data-intensive computing,'' \emph{Proceedings of the Cray User Group}, p. 8, 2016.

Unfortunately, the way pandoc does referencing is fixed to the AST representation, which heavily relies on explicit use of hypertargets to deal with citations directly.
It has the form:
{[}1{]} T. Declerck \emph{et al.}, ``Cori - a system to support data-intensive computing,'' \emph{Proceedings of the Cray User Group}, p. 8, 2016.

Thankfully, we can write a filter using the pandoc-filter python package to update the AST automagically!

The Workflow

First off, it’s pretty common to muck up your python code when putting together these filters.
Unfortunately, the error messages are pretty cryptic / non-existant — I got mostly fd:4: hClose: resource vanished (Broken pipe) or Pandoc died with exitcode "83" during conversion.

Secondly, you can’t use print statements — since it’s included in AST output — or a debugger such as pdb since the process is spawned on a separate process, no stepping through code for you!

A method that I found worked was to generate the intermediate JSON representation used by pandoc’s AST.
This can be generated easily using pandoc -t json and can be stored as a file.
Next, I fired up ipython and installed the pypandoc library — this was really useful for fast iterations to text my filter.
Load up the json file in ipython such as: json_dat = open('test_out.json').read()
Now, you can quickly prototype your filter with pypandoc.convert_text(json_dat,'tex',format='json',filters=[os.path.join('pandoc-tools','')]) — for my filter file called

Checking between the output of the function — just involves checking latex output — to see if the desired changes were made.
Most of my time was spent printing json_dat, skipping to the bad chunk of code, and counting the number of []’s in the AST to figure out why the variable of interest wasn’t collected. Thankfully, these errors around not collecting the right number of arguments are described extensively in the pandoc-filter output!

The Solution

Voila! The result of my bib-filter now generates:
\bibitem{b1}\hypertarget{ref-declerck2016cori}{} T. Declerck \emph{et al.}, ``Cori - a system to support data-intensive computing,'' \emph{Proceedings of the Cray User Group}, p. 8, 2016.

Good luck and happy hacking!
p.s. the pandoc community need’s all the filters we can get, so thank you for viewing this post.

Writing sciency things in Markdown – with Pandoc!

Pandoc is an awesome tool!
This is especially true once properly configured for scientific writing.
I get really distracted writing LaTeX directly — it’s really easy to lose track of what you want to say when writing when you could spend half the day type-setting and resizing figures.
This is where writing in markdown really shines; it allows you the flexibility of LaTeX — since TeX can be embedded at any part of the document — without you going down the long road of type-setting and losing your train of thought.
Best of all, if you’re about to submit the paper and need to finally focus on typesetting it’s easy to generate a LaTeX output of your work and edit as you normally would using the classic TeX workflow.
The full code is available on github and was built with the following packages:

  • pandoc — 1.19.2
  • pandoc-citeproc — 0.10.4
  • pandoc-crossref —

The corresponding pdfs can be viewed here as ACM, IEEE and LNCS.

Personally, I write all my papers in Markdown — or RMarkdown for the fancy stuff that requires generating figures — and leave pandoc to automatically produces pdfs and LaTeX output.
In fact, all my builds are simultaneously generated for 3 separate versions — corresponding to the major style guides in computer science — each in ACM, IEEE and LNCS formatting.

Happy writing!

One Kidney sold every hour for iPads!!

COMPUTER tablets are among the inducements to part with an organ being offered around the world as the illegal trade in kidneys spirals to the point where one is traded every hour.

About 10,000 illegal kidney operations take place every year – about 75 per cent of the black market of all organs traded worldwide – the UK’s Guardian reported, citing Dr. Luc Noel, a World Health Organisation (WHO) official who monitors trends in organ transplants.

The increasing number of people suffering heart problems, diabetes and high blood pressure is pushing the rise in the illicit trade of kidneys.

“There is a growing need for transplants and big profits to be made. It’s ever-growing, it’s a constant struggle,” Noel said.

“The stakes are so big, the profit that can be made so huge, that the temptation is out there.”

WHO says organ brokers prey on the poor and vulnerable, paying them about $5000 for their kidneys.

“Donate a kidney, buy the new iPad!” a broker in China advertised, offering $3920 a kidney.

The organ can later sell for several times the original price, with wealthy patients travelling to China, India or Pakistan for a $200,000 transplant.

The practice is not confined to Third World countries.

Last week, a doctor and nine others were arrested in Israel on suspicion of belonging to an international trafficking ring.

“The people who gain are the rich transplant patients who can afford to buy a kidney, the doctors and hospital administrators, and the middlemen, the traffickers,” said Jim Feehally, a professor at University Hospitals of Leicester NHS Trust.

WHO noted that the total of 106,879 transplants in 2010, which included legal and illegal surgeries, satisfied only 10 per cent of worldwide need.

Read more:

Sorry we confused UN logo with the Halo video game

BBC Halo vs UN

Here’s how the UN was doing it wrong. On the left is the UN logo. On the right, the one from Halo. Source: Supplied

BBC Halo

The BBC displayed a logo from Halo, while newsreader Sophie Raworth presented a report on the deadly unrest in Syria. Picture: BBC Source: Supplied

THE BBC apologised to viewers after mistakenly showing a graphic from the hit computer game Halo in place of the UN Security Council logo.

In a report last week on the continuing deadly unrest in Syria, the British public broadcaster ran a large graphic behind presenter Sophie Raworth as she read the news, The Guardian reported.

Unfortunately, instead of the Security Council logo, viewers were shown the badge of the United Nations Space Command, the military agency depicted in popular interstellar war game Halo.

Viewers quickly spotted the error and posted pictures on social media sites, The Sun reported.

A BBC spokesperson said: “BBC News makes every effort to ensure the accuracy of all images broadcast, however very occasionally mistakes do happen.

“Unfortunately an incorrect logo was used during a segment on last week’s News at One bulletin and we apologise to viewers for the mistake.

“The image was not broadcast in our later bulletins.”

Read more:

Kogan imposes world’s first Internet Explorer 7 tax

kogantaxthumbOnline retailer Kogan has imposed the world’s first tax on customers who use an outdated browser like Internet Explorer 7 for their purchases on its site.

Shoppers who stick with IE7 to purchase products on the recently updated Kogan website will be slugged an additional 6.8 per cent.

But customers who come to the site with IE7 can avoid the tax by downloading an up-to-date browser like a more recent version of IE, Google Chrome, Firefox, Safari or Opera using a pop-up on the site.

Kogan says the company was sick of wasting time ensuring its revamped website worked on Microsoft’s old IE7 browser.

“Internet Explorer 7 has long since passed its use-by date,” says CEO Ruslan Kogan.

The pop-up customers will see if they visit the revamped Kogan site with IE7The pop-up customers will see if they visit the revamped Kogan site with IE7

“It’s a constant source of frustration for our web guys and we’re sick of burning cash on a browser that hit the market nearly six years ago. It goes against everything Kogan stands for.”

“It’s not only costing us a huge amount, it’s affecting any business with an online presence, and costing the internet economy millions of dollars.

Customers who insist on sticking with IE7 wll be slugged an additional 6.8 per cent on their purchaseCustomers who insist on sticking with IE7 wll be slugged an additional 6.8 per cent on their purchase

“Secondly, Kogan has established its market leading pricing by cutting unnecessary costs out of its business model. It makes little sense for us to be working so hard to deliver the best possible prices for electronics, and then being wasteful with our own IT spend.”

The hilarious new 6.8 per cent IE7 tax is effective today but customers can easily avoid the added cost with a simple browser update.

Unique Cat Ear Headphones

Axent Wear Partners with Brookstone to Make and Distribute Their Unique Cat Ear Headphones

In November 2014, Wenqing Yan and Victoria Hu of Axent Wear successfully funded their unique glowing over-ear headphones with adorable cat ear speakers through Indiegogo. Now, one year later, Axent Wear teamed up with the unique gift retailer Brookstone to manufacture, distribute and handle all customer service issues regarding the headphones. The headphones also got a bit of a redesign and are currently available for purchase. Founder Wenqing Yan visited the Brookstone headquarters earlier this year.

This week I was able to visit Brookstone in New Hampshire to personally test out the latest headphones prototype and to give feedback on how to improve for the next prototype. We heard everyone’s feedback on the last post and we completely agree with the changes that need to be made, such as color, brighter lights, and a less bulky frame. Please understand that the prototypes are not what the final product will look like, and we will be actively improving each prototype we make. In this second version of the prototype, we’ve already improved many details, such as the colors and added the detachable gaming boom mic. With each iteration, it will look better and better!

(Via Laughing Squid.)