Is F# development and promotion lagging and ignoring data science?


#1

I was checking the mbrace project (mbrace.io) for cloud computing with F#/C#. The project seems to be either dead or not maintained much. The latest updated ROADMAP is over 3 years old and most of git updates seem to be over one year old, which for a young project does not seem right. While Spark has risen sharply in use and has greatly helped adoption of Scala, MBrace seems to be going the way of the Dodo.
I have always just enjoyed the elegance of coding in F#, but prohibitive issues have remained constantly present over the last 6-7 years. Recently, I was talking about F# to a data scientist and researcher and he posed several questions:

  • Do I have reliable and well-document access to libraries and packages for exploration, visualization, and modeling?
  • Do I have reliable and well-document utilities for doing GPU programming?
  • Do I have reliable and well-document access to tools like Hadoop and Spark?
  • Do I have reliable and well-document tools for utilizing Keras, CNTK, TensorFlow, and alike?

I did not have good answers for any of them. First of all, even when you have something available for F#, documentation and examples are severely lacking. Most of the times though, you don’t get anything native or F# sugar-coated and wrapped. You can use Mobius but it is primarily a C# API. You can use AleaGPU but still documentation and examples are mostly geared for C#. Accord.NET is a very capable package but it still is C#-ish. ML.NET and Encog are very interesting and capable but again C#-oriented. You are even likely to get a Python interface for a Microsoft product much faster than F#.

Another point of worry is the insufficient levels of investment (money and people) by Microsoft. When you have a corporate giant giving birth to and purporting to support a technology, you kind of expect a lot more. Both cloud computing platforms (OneNet/Prajna and MBrace) seem to be goners, and they do not seem to have anything in the pipeline. The simple fact is, people doing things in their free-time for nothing, when the community is not so big and diverse, is simply not going to cut it. I just hope that they develop the willingness to throw some money and resources at critical and valuable projects. Letting go of big data and cloud computing competing with JVM products just does not seem to be very farsighted and wise.
BTW - I really do not see this as an opinion-based topic. Whether or not you like a BMW or whatever brand is an opinion but whether or not it has certain features to make it a safe automobile has a lot more to do with facts. Whether or not there is sufficient investment in a certain product by some company and whether appropriate tooling and support are present and being actively developed in some ecosystem to make it competitive for particular tasks/projects, should not be matters of personal taste and opinion.


#2

ML.net works currently with F#. However you are correct that it’s currently implemented with C# in mind. Being said the team has it as an initiative to better support F# and discussion is being had around how to do that. As for your concerns about Microsoft, I would not worry about them. F# is useful today, it will be actively maintained by the community for the foreseeable future. It is supported in Microsoft’s latest tools and products such as Azure. I wouldn’t depend on Microsoft to dump large resources into F# any time soon but it’s hardly going away. The fact that the language is open means it is here to stay with or without Microsoft. There are even some here who would consider Microsoft’s gentle influence on the language a blessing.


#3

I want to comment a bit on the ‘Microsoft’ part.

Over the last 10 years, Microsoft has made a huge shift from a very corporate-y for-profit entity, to contributing a lot of public effort towards open-source initiatives. Additionally, I don’t think you can really ‘measure’ the resources Microsoft is putting into F#. Sure, we see @cartermp and Don Syme regularly, but Microsoft is not particularly well-known for divulging the entirety of a (the) team. We may not see all the resources it puts in, because it does have internal workings.

Additionally, Microsoft’s involvement is less necessary due to the open-source setup F# follows. By using the RFC process, and open-sourcing the entire compiler and editing tools, Microsoft has allowed those using the language to make first-class changes to it. It’s not longer only considering it’s own interests, it’s actually allowing us, the community and developers using it, to put what we need into the language. I think the biggest reason we don’t see as much public Microsoft involvement is that, I’m 99% sure, they don’t use it internally. C# is more heavily-invested because it is the de-facto tool for Microsoft work, as well as a very standard tool for third-party developers (us).