Programming in VB and Internet text

[MarcusMel]

I have been looking at the Mumbles tutortrial over in the Betfair Free API forum. What I would like to be able to do is programatically get the text of a webpage and process it so that I can combine it with the betfair API. I remember MatR did an Excel progrm to look for system selections and did something similar to what I am asking here.

Any helpful sugestions?

[Win2Win]

Erm...:doh

It's actually pretty easy with vB, just call the web page you want into a string, and then manipulate it which ever way you wish.

http://www.sourcecodester.com/tutorials/visual-basic/5679/visual-basic-webpage-scraper.html

http://blog.hartleybrody.com/web-scraping/

[barneymather]

You're just a bit too blimmin' good with this programming malarkey, Mr Driscoll. What book and language would you recommend for a newbie?

[Win2Win]

Download Visual Basic from Microsoft first and have a little play with it, you can't do any damage. :smile:

The best book is the Dummies Guide to Visual Basic. Simple to follow.

When you need code for this such as 'write to disk', just search the internet, as someone else will have written, so it saves you the hassle. :biggrin:

[MarcusMel]

That was sort of going to be my sugesstion too. (Thanks for the find Keith, the places I looked were unsatisfactory:doh1: )

I would just like to add the following comment to learning any programming language.

You have to first decide on a project of something you want to do. Then pick the language suitable for that project. Then break the project down into the smallest ideas that you can. After that is is just a question of putting the bits together until the project is complete. To try and learn programming because it is there seems pointless to me. I got interested in programming because my brother discussed with me projects that he had been given to program like get the computer to play naughts and crosses, learn to play the game of Nim. Later we discussed the possibilities and ways to interpret ideas of artificial intelligence. This was before the internet was invented.

Every programming language resolves into four basic concepts.
1) Communication (input and output),
2) Conditional action (If {this is true} then {do this} else {do this} )
3) Assignment eg put these numbers in this array
4) Loops repeat a code section until a condition is reached.

[MarcusMel]

Ok small secondary problem with screen scraping Suppose I am looking at the following link

http://www.racingpost.com/horses/trainer_home.sd?trainer_id=10152#topTrainerTabs=trainer_record_race_form&bottomTrainerTabs=trainer_big_race_wins

That is the GB Flat data for Tim Easterby how can I get the GB jumps data? For that matter any of the major overall stats.
I have got a little tired of typing out the Horse name, trainer name, Trainer win frequency for race type and then processing the data for my system test, which over 120 races has shown some success so it may be worth the programming effort.

[Win2Win]

Use another source http://www.attheraces.com/form.aspx?id=1000047&ref=formtrainer&dte=2013-10-20&refsite=&type=t&view=stats

http://www.attheraces.com/search.aspx?query=easterby&type=T

[MarcusMel]

The system might work with overall stats from http://www.sportinglife.com/racing/stats/jump-trainers

Will have to start testing with these figures if I want to have an easy life.

I don't mind copy and pasting all the figures for the Trainers if I have to. The idea is to reduce the amount of work though, not increase it!! :doh

[Win2Win]

If you can figure out from the source code on the RP site what the Java code is calling, you can then retrieve the page yourself. Their is software for monitoring what IE calls, but I've forgotten what it is. :doh1:

[MarcusMel]

Here is some VB code to read a single race off the Racing Post.
I order todays races by time then click on a race of interest. Copy the RP URL into The VB form then click on the race info button and get a copy paste list for excel.

The VB form has two buttons bScrape, bRaceInfo and two text boxes tLinkURL, SrcBox. SrcBox Property set to multi-line for the the results to be printed to.

Might be of interest to someone.
-------------------------------
Imports System.Net
Imports System.IO

Public Class Form1

Private Sub bScrape_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles bScrape.Click
If (Not tLinkURL.Text = Nothing) Then
Dim src As String
src = GetWebText(tLinkURL.Text)
'tLinkURL.Text = tLinkURL.Text.ToLower()

SrcBox.Text = src

End If
End Sub
Sub ShowRace(ByVal src As String)
Dim titleArray() As String
Dim Course() As String
Dim Trainer As String
Dim Horse As String

'Markers
'<title> -means- time and Race course found next ending with | Full Racecard | Racing Post</title>
'title="Full details about this TRAINER"> -means- trainer name </a>
'data-outcome-name=" -means- Name of Horse follows ending with "
src = src.Replace(":", "¬")
src = src.Replace("<", ":")
src = src.Replace(">", ":")
src = src.Replace("""", ":")
titleArray = Split(src, ":")
For i As Integer = 0 To titleArray.Length - 1
Trainer = ""
Horse = ""
If (titleArray(i) = "title") Then
If (titleArray(i + 1) <> "¬") Then
Course = Split(titleArray(i + 1), "|")
Course(0) = Course(0).Replace("¬", ".")
Print(Course(0))
End If
End If
If titleArray(i) = "Full details about this TRAINER" Then
Trainer = titleArray(i + 2)
Trainer = Trainer.Replace("&acute;", "'")
titleArray(0) = Trainer
End If
If titleArray(i) = " data-outcome-name=" Then
Horse = titleArray(i + 1)
Horse = Horse.Replace("&acute;", "'")
End If
If titleArray(i) = "div class=" Then 'Dont read single race card after finding this
If titleArray(i + 1) = "cardFooter" Then
i = titleArray.Length - 1
End If
End If
If Horse <> "" Then
titleArray(0) = Horse + Chr(9) + titleArray(0)
Print(titleArray(0))
End If
Next
Print("")
End Sub
Sub Print(ByVal Message As String)
With SrcBox
.SelectionStart = .Text.Length
.SelectedText = vbCrLf & Message
End With
End Sub

Function GetWebText(ByVal SomeURL As String) As String
If (Not SomeURL = Nothing) Then
SomeURL = SomeURL.ToLower()
If (SomeURL.StartsWith("https://") Or SomeURL.StartsWith("http://")) Then
If (Not SomeURL.StartsWith("https://www.") And Not SomeURL.StartsWith("http://www.")) Then
If Not (SomeURL.StartsWith("www.")) Then
If (SomeURL.StartsWith("http://")) Then
SomeURL = "http://www." & SomeURL.Substring(7, SomeURL.Length - 7)
Else
SomeURL = "https://www." & SomeURL.Substring(8, SomeURL.Length - 8)
End If
End If
End If
ElseIf (SomeURL.StartsWith("www.")) Then
SomeURL = "http://" & SomeURL
Else
SomeURL = "http://www." & SomeURL
End If
Dim req As HttpWebRequest = HttpWebRequest.Create(SomeURL)
Dim res As HttpWebResponse = req.GetResponse()
Dim src As String = New StreamReader(res.GetResponseStream()).ReadToEnd()
Return src
End If
Return ""
End Function


Private Sub bRaceInfo_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles bRaceInfo.Click
If (Not tLinkURL.Text = Nothing) Then
Dim src As String
src = GetWebText(tLinkURL.Text)
ShowRace(src)
End If
End Sub
End Class

[Win2Win]

I was worried you were going to ask a question after all that! :laugh

Although some of the things in programming would seem to be complicated, by the time you have put the code together, their isn't much of it.