Andre's Blog

Andre's Technical Blog About Using Gradle


20160731 About This Technical Blog
20160731 Using Gradle To Build A Jar With Dependencies
20160809 Using Gradle And Groovy To Create An RSS Feed
20161003 Build Script With Topic Support
20161106 Refactored Gradle Build

20160731 About This Technical Blog

To know why I write a blog about using Gradle, please read my blog post

20160731 Using Gradle To Build A Jar With Dependencies

Currently I am struggling with building a Java application with Gradle, which uses groovy-all.2.4.7.jar to execute Groovy code.

I have already explained in this blog post, that it seems to be not enough to just have a compile-dependency to this jar, because when running my application, it complains abaout missing the Groovy classes though these should be available via classpath.

FYI: I have renamed the main class from CISystem to NetworkInitialization for this example.

This was my first Gradle script to build my application :

apply plugin: 'java'

repositories {
    mavenCentral()
}

jar {
    dependencies {
      compile 'org.codehaus.groovy:groovy-all:2.4.7'
    }
    manifest {
        attributes 'Main-Class': 'com.wartbar.cisystem.NetworkInitialization'
    }
}

After googling for a solution and not finding anything I was looking for a way to integrate the Groovy jar into the jar off my application. I have found these two articles, which both present the same solution :

MKYONG: Gradle - Create a Jar file with dependencies
Stackoverflow: Using Gradle to build a jar with dependencies

My build script is now this one :

apply plugin: 'java'

repositories {
    mavenCentral()
}

task fatJar(type: Jar) {
    manifest {
        attributes 'Main-Class': 'com.wartbar.cisystem.NetworkInitialization'
    }
    baseName = project.name + '-all'
    from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
    with jar
}

dependencies {
  compile 'org.codehaus.groovy:groovy-all:2.4.7'
}

This new fatJar task works perfect, it does exactly what I want it to do.

My only problems are :

  1. Why doesn't the jar created by the first script work?
  2. How does the second script work?

I will work on this soon, stay tuned!

20160809 Using Gradle And Groovy To Create An RSS Feed

To find out how my RSS feed looks like and which URL it has, please read my blog post

This is the Gradle/Groovy code in my build.gradle file :

import java.text.SimpleDateFormat
import static java.util.Calendar.*

void writeBeginRss(File f) {
  f << '''<?xml version="1.0" encoding="ISO-8859-1" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<atom:link href="http://www.wartbar.de/rss.xml" rel="self" type="application/rss+xml" />
<title>Wartbar.de RSS-Feed</title>
<link>http://www.wartbar.de</link>
<description>Home of Andre's Blog</description>
<language>de-de</language>
<copyright>2016 by Wartbar.de</copyright>\n'''
}

String[] getSubjectParts(String line) {
  String[] parts = line.split(" ")
  return parts[2..parts.length-1]
}

String getSubject(String line, String fillPattern) {
  String subject = ""
  String[] partsSelected = getSubjectParts(line)
  for (i in partsSelected) {
    subject += i + fillPattern
  }

  return subject[0..-2] // remove last character before returning
}

String getTitle(String line) {
  return "<title>" + getSubject(line," ") + "</title>\n"
}

String getSubjectUrl(String line) {
  String subjectUrl = "http://www.wartbar.de/index.html#" + getSubject(line,"-")
  return subjectUrl.toLowerCase()
}

String getLink(String line) {
  return "<link>" + getSubjectUrl(line) + "</link>\n"
}

String getGuid(String line) {
  return "<guid>" + getSubjectUrl(line) + "</guid>\n"
}

String getDescription(String line) {
  return "<description>A new blog post with the subject '" +
          getSubject(line," ") +
          "'</description>\n"
}

String getRFC882DateString(String line) {

  String input = line.split(" ")[1]

  logger.info input[0..3] + ":" + input[4..5] + ":" + input [6..7]

  int year = Integer.parseInt(input[0..3])
  int month = Integer.parseInt(input[4..5])-1
  int day = Integer.parseInt(input[6..7])

  Calendar cal = Calendar.instance
  cal.set year, month, day, 0, 0, 0
  SimpleDateFormat sdf = new SimpleDateFormat( 'EEE, d MMM yyyy HH:mm:ss Z', Locale.US )
  return sdf.format( cal.getTime() )
}

String getPubDate(String line) {
  return "<pubDate>" + getRFC882DateString(line) + "</pubDate>\n"
}

void writeItem(File f, String line) {
  f << "<item>\n"
  f << getTitle(line)
  f << getLink(line)
  f << getDescription(line)
  f << getGuid(line)
  f << getPubDate(line)
  f << "</item>\n"
}

String writeEndRss(File f) {
  f << "</channel>\n"
  f << "</rss>\n"
}

task genRSS(group: "Blog") {
  description = "generates the RSS feed for this blog based on the <h3> tags"

  doLast {
    File f = file("rss.xml")
    f.delete()

    writeBeginRss(f)

    def lines = file("blog.md").readLines()

    for (line in lines) {
      if (line.startsWith("### 20")) {
        writeItem(f, line)
      }
    }

    writeEndRss(f)
  }
}

20161003 Build Script With Topic Support

The build script now

and these are the sources which have helped me :

Joerg Mueller: Executing shell commands in Groovy
Gradle Userguide: Working With Files

import java.text.SimpleDateFormat
import static java.util.Calendar.*

void writeBeginRss(File f) {
  f << '''<?xml version="1.0" encoding="ISO-8859-1" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<atom:link href="http://www.wartbar.de/rss.xml" rel="self" type="application/rss+xml" />
<title>Wartbar.de RSS-Feed</title>
<link>http://www.wartbar.de</link>
<description>Home of Andre's Blog</description>
<language>de-de</language>
<copyright>2016 by Wartbar.de</copyright>\n'''
}

String[] getSubjectParts(String line) {
  String[] parts = line.split(" ")
  return parts[2..parts.length-1]
}

String getSubject(String line, String fillPattern) {
  String subject = ""
  String[] partsSelected = getSubjectParts(line)
  for (i in partsSelected) {
    subject += i + fillPattern
  }

  return subject[0..-2] // remove last character before returning
}

String getTitle(String line) {
  return "<title>" + getSubject(line," ") + "</title>\n"
}

String getSubjectUrl(String line) {
  String subjectUrl = "http://www.wartbar.de/index.html#" + getSubject(line,"-")
  return subjectUrl.toLowerCase()
}

String getLink(String line) {
  return "<link>" + getSubjectUrl(line) + "</link>\n"
}

String getGuid(String line) {
  return "<guid>" + getSubjectUrl(line) + "</guid>\n"
}

String getDescription(String line) {
  return "<description>A new blog entry with the subject '" +
          getSubject(line," ") +
          "'</description>\n"
}

String getRFC882DateString(String line) {

  String input = line.split(" ")[1]

  logger.info input[0..3] + ":" + input[4..5] + ":" + input [6..7]

  int year = Integer.parseInt(input[0..3])
  int month = Integer.parseInt(input[4..5])-1
  int day = Integer.parseInt(input[6..7])

  Calendar cal = Calendar.instance
  cal.set year, month, day, 0, 0, 0
  SimpleDateFormat sdf = new SimpleDateFormat( 'EEE, d MMM yyyy HH:mm:ss Z', Locale.US )
  return sdf.format( cal.getTime() )
}

String getPubDate(String line) {
  return "<pubDate>" + getRFC882DateString(line) + "</pubDate>\n"
}

void writeItem(File f, String line) {
  f << "<item>\n"
  f << getTitle(line)
  f << getLink(line)
  f << getDescription(line)
  f << getGuid(line)
  f << getPubDate(line)
  f << "</item>\n"
}

String writeEndRss(File f) {
  f << "</channel>\n"
  f << "</rss>\n"
}

task genRSS(group: "Blog") {
  description = "generates the RSS feed for this blog based on the <h3> tags"

  doLast {
    File f = file("rss.xml")
    f.delete()

    writeBeginRss(f)

    def lines = file("blog.md").readLines()

    for (line in lines) {
      if (line.startsWith("### 20")) {
        writeItem(f, line)
      }
    }

    writeEndRss(f)
  }
}

String getPost(String line) {
  String[] parts = line.split(" ")
  String post = ""

  parts[2..parts.length-1].each {
    post += "${it} "
  }

  post = post[0..post.size()-2]

  return post
}

String getTopic(String line) {
  String tmp = line.replaceAll("\\[", "").replaceAll("\\]","");
  return tmp
}

void add(def map, String line, String lastPost) {
  String topic = getTopic(line)
  if (map[(topic)] == null) {
    def posts = []
    posts.add(lastPost)
    map.put((topic), posts)
  } else {
    def posts = map[(topic)]
    posts << lastPost
  }
}

void writeTopicHeader(File f, String topic) {

  f << '''<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" type="text/css" href="../formate.css">
<link rel="alternate" type="application/rss+xml" title="RSS" href="http://www.wartbar.de/rss.xml">
</head>
<header id="header">
<br>
<h1>'''

  f << topic

  f << '''</h1>
<br>
<a href="../menu.html"><img src="../menu.jpg" width="50" ></a>
<br>
<br>
</header>
<section id="section">
<ul>\n'''
}

String getID(String post) {
  return post.toLowerCase().replaceAll(" ", "-")
}

void writePost(File f, String post) {
  f << '<li>['
  f << post
  f << '](../index.html#'
  f << getID(post)
  f << ')\n'
}

void writeTopicFooter(File f) {
  f << '''</ul>
</section>\n'''
}

void createTopicFiles(def map) {
  map.each {

    File f = new File("topics/${it.key}.md")
    f.delete()

    writeTopicHeader(f,"Andre's Blog Posts About ${it.key}")

    def posts = map[(it.key)]
    posts[posts.size()-1..0].each {
      writePost(f,it)
    }

    writeTopicFooter(f)
  }
}

void writeTopic(File f, String topic) {
  f << '<li>['
  f << topic
  f << ']('
  f << topic
  f << '.html)\n'
}

void createTopicMenu(def map) {
  File f = new File("topics/topics.md")
  f.delete()

  writeTopicHeader(f,"Andre's Blog Post Topics")

  map.each {
    writeTopic(f,it.key)
  }
  writeTopicFooter(f)
}

void fillTopicMap(def map, def lines) {
  String lastPost

  for (line in lines) {
    if (line.startsWith("### 20")) {
      lastPost = getPost(line)
    }

    if (line.startsWith("[[")) {
      add(map,line,lastPost)
    }
  }
}

void compileTags(def lines) {
  File f = new File("blogCompiled.md")
  f.delete()
  lines.each {
    if (it.startsWith("[[")) {
      String topic = getTopic(it)
      f << '[['
      f << topic
      f << ']](topics/'
      f << topic
      f << '.html)\n'
    } else {
      f << it << '\n'
    }
  }
}

task genTopics(group: "Blog") {
  doLast {
    def map = [:]
    def lines = file("blog.md").readLines()

    fillTopicMap(map, lines)
    compileTags(lines)
    createTopicFiles(map)
    createTopicMenu(map)
  }
}

def executeOnShell(String command) {
  return executeOnShell(command, new File(System.properties.'user.dir'))
}

def executeOnShell(String command, File workingDir) {
  println command
  def process = new ProcessBuilder(addShellPrefix(command))
                                    .directory(workingDir)
                                    .redirectErrorStream(true)
                                    .start()
  process.inputStream.eachLine {println it}
  process.waitFor();
  return process.exitValue()
}

def addShellPrefix(String command) {
  def commandArray = new String[3]
  commandArray[0] = "bash"
  commandArray[1] = "-c"
  commandArray[2] = command
  return commandArray
}

task mdAll() {
  doLast {
    FileTree tree = fileTree(dir: '.', include: '**/*.md')
    tree.each {File file ->
      String fileName = file.getName()
      String name = fileName[0..fileName.size()-4]
      String command = "pandoc " + file.getAbsolutePath() + " > " + file.getParent() + "/" + name + ".html"
      executeOnShell(command)
    }
  }
}

task createIndexHtml(group: "Blog") {
  doLast {
    executeOnShell("pandoc blogCompiled.md > index.html")
  }
}

task compileMD(group: "Blog") {
  dependsOn genRSS
  dependsOn genTopics
  dependsOn mdAll
  dependsOn createIndexHtml
}
mdAll.mustRunAfter(genTopics)
createIndexHtml.mustRunAfter(mdAll)

20161106 Refactored Gradle Build

Adding new functionality to my gradle build has increased its code and has made it hard read read.

For this reason I have started to refactor it.

I have decided to

  1. separate the code into different Groovy classes
  2. move the code to the buildSrc
  3. have custom Gradle tasks for those classes where I don't need to pass parameters

This is my new build.gradle script :

apply plugin: 'groovy'

dependencies {
    compile ('org.codehaus.groovy:groovy-all:2.4.6',)

    testCompile(
            'junit:junit:4.12',
            'org.codehaus.groovy:groovy-all:2.4.6',
            'org.spockframework:spock-core:1.0-groovy-2.4',
    )

    testRuntime(
            'com.athaydes:spock-reports:1.2.7'
    )
}

import com.wartbar.topic.TagCompilerTask
import com.wartbar.topic.TopicMap
import com.wartbar.topic.TopicFileGenerator
import com.wartbar.topic.TopicMenuGenerator
import com.wartbar.rss.RSSGeneratorTask
import com.wartbar.util.ShellExecute

task genRSS(type: RSSGeneratorTask, group: "Blog") {
  description = "generates the RSS feed for this blog based on the <h3> tags"

  sourceFile = new File("blog.md")
  destinationFile = new File("rss.xml")
}

task makeCompiledBlog(type: TagCompilerTask) {
  sourceFile = new File("blog.md")
  destinationFile = new File("blogCompiled.md")
}

task genTopics(group: "Blog") {

  dependsOn makeCompiledBlog

  doLast {

    TopicMap tm = new TopicMap(file("blog.md"))
    def map = tm.accessMap()

    TopicFileGenerator tfg = new TopicFileGenerator()
    tfg.create(map)

    TopicMenuGenerator tmg = new TopicMenuGenerator()
    tmg.create(map)
  }
}

task mdAll() {
  doLast {
    FileTree tree = fileTree(dir: '.', include: '**/*.md')
    tree.each {File file ->
      String fileName = file.getName()
      String name = fileName[0..fileName.size()-4]
      String command = "pandoc " + file.getAbsolutePath() + " > " + file.getParent() + "/" + name + ".html"
      ShellExecute.exec(command)
    }
  }
}

task createIndexHtml(group: "Blog") {
  doLast {
      ShellExecute.exec("pandoc blogCompiled.md > index.html")
  }
}

task compileMD(group: "Blog") {
  dependsOn genRSS
  dependsOn genTopics
  dependsOn mdAll
  dependsOn createIndexHtml
}
mdAll.mustRunAfter(genTopics)
createIndexHtml.mustRunAfter(mdAll)

This is the RSSGeneratorTask class :

package com.wartbar.rss

import org.gradle.api.DefaultTask
import org.gradle.api.tasks.TaskAction
import org.gradle.api.tasks.InputFile
import org.gradle.api.tasks.OutputFile

import java.text.SimpleDateFormat

class RSSGeneratorTask extends DefaultTask{

    @InputFile
    File sourceFile

    @OutputFile
    File destinationFile


    void writeBeginRss(File f) {
        f << '''<?xml version="1.0" encoding="ISO-8859-1" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<atom:link href="http://www.wartbar.de/rss.xml" rel="self" type="application/rss+xml" />
<title>Wartbar.de RSS-Feed</title>
<link>http://www.wartbar.de</link>
<description>Home of Andre's Blog</description>
<language>de-de</language>
<copyright>2016 by Wartbar.de</copyright>\n'''
    }

    String[] getSubjectParts(String line) {
        String[] parts = line.split(" ")
        return parts[2..parts.length-1]
    }

    String getSubject(String line, String fillPattern) {
        String subject = ""
        String[] partsSelected = getSubjectParts(line)
        for (i in partsSelected) {
            subject += i + fillPattern
        }

        return subject[0..-2] // remove last character before returning
    }

    String getTitle(String line) {
        return "<title>" + getSubject(line," ") + "</title>\n"
    }

    String getSubjectUrl(String line) {
        String subjectUrl = "http://www.wartbar.de/index.html#" + getSubject(line,"-")
        return subjectUrl.toLowerCase()
    }

    String getLink(String line) {
        return "<link>" + getSubjectUrl(line) + "</link>\n"
    }

    String getGuid(String line) {
        return "<guid>" + getSubjectUrl(line) + "</guid>\n"
    }

    String getDescription(String line) {
        return "<description>A new blog entry with the subject '" +
                getSubject(line," ") +
                "'</description>\n"
    }

    String getRFC882DateString(String line) {

        String input = line.split(" ")[1]

        logger.info input[0..3] + ":" + input[4..5] + ":" + input [6..7]

        int year = Integer.parseInt(input[0..3])
        int month = Integer.parseInt(input[4..5])-1
        int day = Integer.parseInt(input[6..7])

        Calendar cal = Calendar.instance
        cal.set year, month, day, 0, 0, 0
        SimpleDateFormat sdf = new SimpleDateFormat( 'EEE, d MMM yyyy HH:mm:ss Z', Locale.US )
        return sdf.format( cal.getTime() )
    }

    String getPubDate(String line) {
        return "<pubDate>" + getRFC882DateString(line) + "</pubDate>\n"
    }

    void writeItem(File f, String line) {
        f << "<item>\n"
        f << getTitle(line)
        f << getLink(line)
        f << getDescription(line)
        f << getGuid(line)
        f << getPubDate(line)
        f << "</item>\n"
    }

    String writeEndRss(File f) {
        f << "</channel>\n"
        f << "</rss>\n"
    }

    @TaskAction
    public generate() {

        destinationFile.delete()

        writeBeginRss(destinationFile)

        def lines = sourceFile.readLines()

        for (line in lines) {
            if (line.startsWith("### 20")) {
                writeItem(destinationFile, line)
            }
        }

        writeEndRss(destinationFile)
    }
}

This is the TagCompilerTask :

package com.wartbar.topic

import org.gradle.api.DefaultTask
import org.gradle.api.tasks.TaskAction
import org.gradle.api.tasks.InputFile
import org.gradle.api.tasks.OutputFile

class TagCompilerTask extends DefaultTask {

    @InputFile
    File sourceFile

    @OutputFile
    File destinationFile

    String getTopic(String line) {
        return line.replaceAll("\\[", "").replaceAll("\\]","")
    }

    @TaskAction
    public void compile() {

        def lines = sourceFile.readLines()

        destinationFile.delete()
        lines.each {
            if (it.startsWith("[[")) {
                String topic = getTopic(it)
                destinationFile << '[['
                destinationFile << topic
                destinationFile << ']](topics/'
                destinationFile << topic.replace(" ","_")
                destinationFile << '.html)\n'
            } else {
                destinationFile << it << '\n'
            }
        }

    }
}

This is the TopicFileGenerator class :

package com.wartbar.topic

class TopicFileGenerator {

    public TopicFileGenerator() {}

    void writeTopicFooter(File f) {
        f << '''</ul>
</section>\n'''
    }

    String getID(String post) {
        return post.toLowerCase().replaceAll(" ", "-")
    }

    void writePost(File f, String post) {
        f << '<li>['
        f << post
        f << '](../index.html#'
        f << getID(post)
        f << ')\n'
    }

    void writeTopicHeader(File f, String topic) {

        f << '''<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" type="text/css" href="../formate.css">
<link rel="alternate" type="application/rss+xml" title="RSS" href="http://www.wartbar.de/rss.xml">
</head>
<header id="header">
<br>
<h1>'''

        f << topic

        f << '''</h1>
<br>
<a href="../menu.html"><img src="../menu.jpg" width="50" ></a>
<br>
<br>
</header>
<section id="section">
<ul>\n'''
    }

    void create(def map) {
        map.each {

            String filename = it.key.replace(" ","_")
            File f = new File("topics/${filename}.md")
            f.delete()

            writeTopicHeader(f,"Andre's Blog Posts About ${it.key}")

            def posts = map[(it.key)]
            posts[posts.size()-1..0].each {
                writePost(f,it)
            }

            writeTopicFooter(f)
        }
    }
}

This is the TopicMenuGenerator class

package com.wartbar.topic

class TopicMenuGenerator {

    public TopicMenuGenerator() {}

    void writeTopicListFooter(File f) {
        f << '''</section>\n'''
    }

    void writeData(File f, String x) {
        f << '<td>'

        if (!x.isEmpty()) {
            f << '<a class="paddedTable" href="'
            f << x.replace(" ","_")
            f << '.html">'
            f << x
            f << '</a>'
        }

        f << '</td>\n'
    }

    void writeTopicRow(File f, String a, String b, String c, String d) {
        f << '<tr>\n'

        writeData(f,a)
        writeData(f,b)
        writeData(f,c)
        writeData(f,d)

        f << '</tr>\n'
    }

    String element(ArrayList<String> list, int index) {
        if (index < list.size()) {
            return list.get(index)
        } else {
            return ""
        }
    }

    void writeTopicListHeader(File f, String topic) {

        f << '''<!DOCTYPE html>
<html style="background-color:black;text-align:center;">
<head>
<link rel="stylesheet" type="text/css" href="../formate.css">
<link rel="alternate" type="application/rss+xml" title="RSS" href="http://www.wartbar.de/rss.xml">
</head>
<br>
<h1>'''

        f << topic

        f << '''</h1>
<br>
<a href="../menu.html"><img src="../menu.jpg" width="50" ></a>
<br>
<br>
<section id="section">'''
    }

    void create(def map) {
        File f = new File("topics/topics.md")
        f.delete()

        writeTopicListHeader(f,"Andre's Blog Post Topics")

        ArrayList<String> orderedList = new ArrayList<>()
        orderedList.addAll(map.keySet())
        Collections.sort(orderedList)

        println "Topic Count : " + orderedList.size()

        f << '<table class="paddedTable">\n'

        int divider = (orderedList.size() / 4) + 1

        for (int i=0; i<divider; i++) {
            String a = element(orderedList, i)
            String b = element(orderedList, i + divider)
            String c = element(orderedList, i + (2*divider))
            String d = element(orderedList, i + (3*divider))

            writeTopicRow(f, a, b, c, d )
        }

        f << "</table>\n"

        writeTopicListFooter(f)
    }
}

This is the TopicMap class :

package com.wartbar.topic

class TopicMap {

    def map = [:]

    def accessMap() {
        return map
    }

    String getTopic(String line) {
        return line.replaceAll("\\[", "").replaceAll("\\]","")
    }

    String getPost(String line) {
        String[] parts = line.split(" ")
        String post = ""

        parts[2..parts.length-1].each {
            post += "${it} "
        }

        post = post[0..post.size()-2]

        return post
    }

    void add(String line, String lastPost) {
        String topic = getTopic(line)
        if (map[(topic)] == null) {
            def posts = []
            posts.add(lastPost)
            map.put((topic), posts)
        } else {
            def posts = map[(topic)]
            posts << lastPost
        }
    }

    public TopicMap(File inputFile) {
        def lines = inputFile.readLines()
        String lastPost

        for (line in lines) {
            if (line.startsWith("### 20")) {
                lastPost = getPost(line)
            }

            if (line.startsWith("[[")) {
                add(line,lastPost)
            }
        }
    }
}

This is the ShellExecute class :

package com.wartbar.util

class ShellExecute {

    public static int exec(String command) {
        return exec(command, new File(System.properties.'user.dir'))
    }

    public static int exec(String command, File workingDir) {
        println command
        def process = new ProcessBuilder(addShellPrefix(command))
                .directory(workingDir)
                .redirectErrorStream(true)
                .start()
        process.inputStream.eachLine {println it}
        process.waitFor();
        return process.exitValue()
    }

    public static String[] addShellPrefix(String command) {
        def commandArray = new String[3]
        commandArray[0] = "bash"
        commandArray[1] = "-c"
        commandArray[2] = command
        return commandArray
    }
}

Select where to go ...


The Blog
My Technical Blogs
Projects
Blogs Of Friends
RSS
CV/About
Me