multiple text files mining

hey everbody,

i'm absolutely new to any sort of data management

here it goes: suppose we store 100 .txt or .doc files in sql server and we want that none of the files data should match more than 60%: the question which arises are

1. how do we store files in ms-sql (binary format or normal text)?

2. how do we match the files?

3. what code we write in c# for this purpose?

4. has this nething to do with pattern recognition?

My request to all new n active experienced user's to participate because Plzzzzz help me?

1.I should use a SSIS solution using "Import Column transform" to get the files and store in a column with varbinary(max) data type; if you want to store only text in a column you can use varchar(max) data type (you can store maximum 2 GB)

2.Using SSIS solution I told there is a way to matching file (use "for each loop container")

3.For my ideea you don't to do that; you can run periodically the package created with SSIS in a job , depends on your business logic

4.Study the tutorial from here if you want create a text mining project. The ideea if you want to know let say the frequence of the terms /phrases, extracting clustering terms/concepts from the docs.

|||

does SSIS have a readymade package to compare mulitple files? if yes how does one go about it? thanx man !!!

|||

what did ou mean "compare multiple files"?

using for each loop container ou can select *.txt or *.doc files or have a special name.

if mean comparing contents of files i think you can use script task when you can customize this comparison using .net (as you should did it using .NET)

|||

i mean that for example there are two text files (.txt or .doc) stored in SQL server, containing an essay on American Independance:

I want to check that the essays do not match more than 60%. How do i do this? help appreciated !!!

|||

vickwal wrote:

i mean that for example there are two text files (.txt or .doc) stored in SQL server, containing an essay on American Independance:

I want to check that the essays do not match more than 60%. How do i do this? help appreciated !!!

You can use MS Integration Service.

I think you have-to convert (presumably MS Word) .doc format into plain text .txt.

Then you can load .txt files into a table like ESSAYS(AUTHOR varchar(255), FILENAME varchar(255), [Content] TEXT) using For Each Loop Container control.

Then use Fuzzy Lookup comparing by Content field, using same ESSAYS table as base and as Lookup table.

You can play with Similarity Threshold there.

Fuzzy Lookup operator will produce an output for each row of base table where Similarity and Confidence columns will be. Just spool it into another table.

good luck,

Mark

multiple text files mining

hey everbody,

i'm absolutely new to any sort of data management

here it goes: suppose we store 100 .txt or .doc files in sql server and we want that none of the files data should match more than 60%: the question which arises are

1. how do we store files in ms-sql (binary format or normal text)?

2. how do we match the files?

3. what code we write in c# for this purpose?

4. has this nething to do with pattern recognition?

My request to all new n active experienced user's to participate because Plzzzzz help me?

1.I should use a SSIS solution using "Import Column transform" to get the files and store in a column with varbinary(max) data type; if you want to store only text in a column you can use varchar(max) data type (you can store maximum 2 GB)

2.Using SSIS solution I told there is a way to matching file (use "for each loop container")

3.For my ideea you don't to do that; you can run periodically the package created with SSIS in a job , depends on your business logic

4.Study the tutorial from here if you want create a text mining project. The ideea if you want to know let say the frequence of the terms /phrases, extracting clustering terms/concepts from the docs.

|||

does SSIS have a readymade package to compare mulitple files? if yes how does one go about it? thanx man !!!

|||

what did ou mean "compare multiple files"?

using for each loop container ou can select *.txt or *.doc files or have a special name.

if mean comparing contents of files i think you can use script task when you can customize this comparison using .net (as you should did it using .NET)

|||

i mean that for example there are two text files (.txt or .doc) stored in SQL server, containing an essay on American Independance:

I want to check that the essays do not match more than 60%. How do i do this? help appreciated !!!

|||

vickwal wrote:

i mean that for example there are two text files (.txt or .doc) stored in SQL server, containing an essay on American Independance:

I want to check that the essays do not match more than 60%. How do i do this? help appreciated !!!

You can use MS Integration Service.

I think you have-to convert (presumably MS Word) .doc format into plain text .txt.

Then you can load .txt files into a table like ESSAYS(AUTHOR varchar(255), FILENAME varchar(255), [Content] TEXT) using For Each Loop Container control.

Then use Fuzzy Lookup comparing by Content field, using same ESSAYS table as base and as Lookup table.

You can play with Similarity Threshold there.

Fuzzy Lookup operator will produce an output for each row of base table where Similarity and Confidence columns will be. Just spool it into another table.

good luck,

Mark

multiple text files mining

hey everbody,

i'm absolutely new to any sort of data management

here it goes: suppose we store 100 .txt or .doc files in sql server and we want that none of the files data should match more than 60%: the question which arises are

1. how do we store files in ms-sql (binary format or normal text)?

2. how do we match the files?

3. what code we write in c# for this purpose?

4. has this nething to do with pattern recognition?

My request to all new n active experienced user's to participate because Plzzzzz help me?

1.I should use a SSIS solution using "Import Column transform" to get the files and store in a column with varbinary(max) data type; if you want to store only text in a column you can use varchar(max) data type (you can store maximum 2 GB)

2.Using SSIS solution I told there is a way to matching file (use "for each loop container")

3.For my ideea you don't to do that; you can run periodically the package created with SSIS in a job , depends on your business logic

4.Study the tutorial from here if you want create a text mining project. The ideea if you want to know let say the frequence of the terms /phrases, extracting clustering terms/concepts from the docs.

|||

does SSIS have a readymade package to compare mulitple files? if yes how does one go about it? thanx man !!!

|||

what did ou mean "compare multiple files"?

using for each loop container ou can select *.txt or *.doc files or have a special name.

if mean comparing contents of files i think you can use script task when you can customize this comparison using .net (as you should did it using .NET)

|||

i mean that for example there are two text files (.txt or .doc) stored in SQL server, containing an essay on American Independance:

I want to check that the essays do not match more than 60%. How do i do this? help appreciated !!!

|||

vickwal wrote:

i mean that for example there are two text files (.txt or .doc) stored in SQL server, containing an essay on American Independance:

I want to check that the essays do not match more than 60%. How do i do this? help appreciated !!!

You can use MS Integration Service.

I think you have-to convert (presumably MS Word) .doc format into plain text .txt.

Then you can load .txt files into a table like ESSAYS(AUTHOR varchar(255), FILENAME varchar(255), [Content] TEXT) using For Each Loop Container control.

Then use Fuzzy Lookup comparing by Content field, using same ESSAYS table as base and as Lookup table.

You can play with Similarity Threshold there.

Fuzzy Lookup operator will produce an output for each row of base table where Similarity and Confidence columns will be. Just spool it into another table.

good luck,

Mark

multiple text docs evaluation

hey everbody,

i'm absolutely new to any sort of data management

here it goes: suppose we store 100 .txt or .doc files in sql server and we want that none of the files data should match more than 60%: the question which arises are

1. how do we store files in ms-sql (binary format or normal text)?

2. how do we match the files?

3. what code we write in c# for this purpose?

4. has this nething to do with pattern recognition?

My request to all new n active experienced user's to participate because Plzzzzz help me?

What is the purpose of this? If you want to discriminate against files, that should be done before saving them. If you need a difference analyzer, I suggest you look how subversion, cvs or any other source control systems handles and saves the differences.|||

the purpose is:

say i store two text documents uploaded to SQL Server from a web portal. Now i want SQL Server to determine that how much percent the content of these two files matches? Say if I want that the files shouldn't match more than 60%, and if they do they should be discarded. thanks man, help appreciated !!!

|||That sounds like a trigger might be able to do that processing during a table load/update. I will move this post to the DB forum for advice.|||thanx man! I was wondering if SQL Server 05 had some SSIP analytic readymade service do that?

multiple text docs evaluation

hey everbody,

i'm absolutely new to any sort of data management

here it goes: suppose we store 100 .txt or .doc files in sql server and we want that none of the files data should match more than 60%: the question which arises are

1. how do we store files in ms-sql (binary format or normal text)?

2. how do we match the files?

3. what code we write in c# for this purpose?

4. has this nething to do with pattern recognition?

My request to all new n active experienced user's to participate because Plzzzzz help me?

What is the purpose of this? If you want to discriminate against files, that should be done before saving them. If you need a difference analyzer, I suggest you look how subversion, cvs or any other source control systems handles and saves the differences.|||

the purpose is:

say i store two text documents uploaded to SQL Server from a web portal. Now i want SQL Server to determine that how much percent the content of these two files matches? Say if I want that the files shouldn't match more than 60%, and if they do they should be discarded. thanks man, help appreciated !!!

|||That sounds like a trigger might be able to do that processing during a table load/update. I will move this post to the DB forum for advice.|||thanx man! I was wondering if SQL Server 05 had some SSIP analytic readymade service do that?

multiple text docs evaluation

hey everbody,

i'm absolutely new to any sort of data management

here it goes: suppose we store 100 .txt or .doc files in sql server and we want that none of the files data should match more than 60%: the question which arises are

1. how do we store files in ms-sql (binary format or normal text)?

2. how do we match the files?

3. what code we write in c# for this purpose?

4. has this nething to do with pattern recognition?

My request to all new n active experienced user's to participate because Plzzzzz help me?

What is the purpose of this? If you want to discriminate against files, that should be done before saving them. If you need a difference analyzer, I suggest you look how subversion, cvs or any other source control systems handles and saves the differences.|||

the purpose is:

say i store two text documents uploaded to SQL Server from a web portal. Now i want SQL Server to determine that how much percent the content of these two files matches? Say if I want that the files shouldn't match more than 60%, and if they do they should be discarded. thanks man, help appreciated !!!

|||That sounds like a trigger might be able to do that processing during a table load/update. I will move this post to the DB forum for advice.|||thanx man! I was wondering if SQL Server 05 had some SSIP analytic readymade service do that?

Multiple Sorting

Hello All!
My report has columns Date, Year, Model, Cost, Color
I have users that will require the abililty to sort by multiple criteria.
Is it possible to configure a primary AND secondary AND maybe a THIRD sort
in Reporting Services?
For example: Sort the above columns as: Date, Model and Cost at the same
time?
This sorting criteri will change based on the user and the info sought. So
another user may want to
sort on Cost and Year, while another person wants Color,Cost, Year etc..
Currently, as far as I know, Reporting Services limits one to one column
sort at a time in a report.
Thanks in advance !
ScottOn Nov 16, 9:53 am, iamscott <iamsc...@.discussions.microsoft.com>
wrote:
> Currently, as far as I know, Reporting Services limits one to one column
> sort at a time in a report.
By default, all of your objects will be populated in the order of the
data in the DataSet. You can define this with a ORDER BY clause in
your SQL.
But, you can create Sort rules by object in your Report. Go to your
Table Properties, then the Sort tab, and you can add multiple sorting
rules to your table. Each rule is applied in sequence, with an
Ascending or Descending. Note that the sorting value is actually an
Expression, which means you can make it dynamic with a little code.
For your problem, what I would do is create a Parameter called
SortRules, make it a Integer, then define Available values as:
"Date, Model, Cost", 1
"Cost, Year", 2
"Color, Cost, Year", 3
etc
Then, in your table, add three Sort rules, each Ascending
Rule 1:
= IIF( Parameters!SortRules.Value = 1, Fields!Date.Value,
IIF( Parameters!SortRules.Value = 2, Fields!Cost.Value,
IIF( Parameters!SortRules.Value = 3, Fields!Color.Value,
Nothing ) ) )
Rule 2:
= IIF( Parameters!SortRules.Value = 1, Fields!Model.Value,
IIF( Parameters!SortRules.Value = 2, Fields!Year.Value,
IIF( Parameters!SortRules.Value = 3, Fields!Cost.Value,
Nothing ) ) )
Rule 3:
= IIF( Parameters!SortRules.Value = 1, Fields!Cost.Value,
IIF( Parameters!SortRules.Value = 2, Nothing,
IIF( Parameters!SortRules.Value = 3, Fields!Year.Value,
Nothing ) ) )
Now, when the user views the report, they will get a Parameter Prompt
at the top to select a sort method, then after choosing one and
hitting the View Report, the table will sort based on that method.
-- Scott|||Thanks so much for your idea! have a great weekend!
Scott
"Orne" wrote:
> On Nov 16, 9:53 am, iamscott <iamsc...@.discussions.microsoft.com>
> wrote:
> > Currently, as far as I know, Reporting Services limits one to one column
> > sort at a time in a report.
> By default, all of your objects will be populated in the order of the
> data in the DataSet. You can define this with a ORDER BY clause in
> your SQL.
> But, you can create Sort rules by object in your Report. Go to your
> Table Properties, then the Sort tab, and you can add multiple sorting
> rules to your table. Each rule is applied in sequence, with an
> Ascending or Descending. Note that the sorting value is actually an
> Expression, which means you can make it dynamic with a little code.
> For your problem, what I would do is create a Parameter called
> SortRules, make it a Integer, then define Available values as:
> "Date, Model, Cost", 1
> "Cost, Year", 2
> "Color, Cost, Year", 3
> etc
> Then, in your table, add three Sort rules, each Ascending
> Rule 1:
> = IIF( Parameters!SortRules.Value = 1, Fields!Date.Value,
> IIF( Parameters!SortRules.Value = 2, Fields!Cost.Value,
> IIF( Parameters!SortRules.Value = 3, Fields!Color.Value,
> Nothing ) ) )
> Rule 2:
> = IIF( Parameters!SortRules.Value = 1, Fields!Model.Value,
> IIF( Parameters!SortRules.Value = 2, Fields!Year.Value,
> IIF( Parameters!SortRules.Value = 3, Fields!Cost.Value,
> Nothing ) ) )
> Rule 3:
> = IIF( Parameters!SortRules.Value = 1, Fields!Cost.Value,
> IIF( Parameters!SortRules.Value = 2, Nothing,
> IIF( Parameters!SortRules.Value = 3, Fields!Year.Value,
> Nothing ) ) )
> Now, when the user views the report, they will get a Parameter Prompt
> at the top to select a sort method, then after choosing one and
> hitting the View Report, the table will sort based on that method.
> -- Scott
>

Multiple Sort Order Specification in EDB database

Hi,

I am using the EDB as a database in smart phone applications.I can able to mount the database , create the tables , write the records into the tables and read from the table with single sort order specification.But if i am using more than one sort order specification seek database is throwing an error message "The drive cannot locate a specific area or track on the disk".

I created one table named as Icon with more than one sort order specification.In this table i am giving pageid and iconid as sort order specifications. While saving the data into the database i am opening the database with pageid sort order and storing into the table.while reading from the database i am opening the database with iconid as sortorder.So here while seeking the database the above error is displaying.And i tried to read the data from the table by using pagid as sortorder speicification.It is working fine.I am applying the 3 sort order specifications for another table named as contentTable. but it is working fine with 3 sort order specifications.I had done the samething for icon table .But no use.Please help me regarding this.

Thanks in Advance,

Thanks & Regards,

Prasanna Kumar

Moving to Sql Server compact edition forum where it has got better chances of being answered.

-Thanks,

Mohit