携程景点数据抓取及分析系统的设计与实现

 博主介绍：✌在职Java研发工程师、专注于程序设计、源码分享、技术交流、专注于Java技术领域和毕业设计✌

项目名称

携程景点数据抓取及分析系统的设计与实现

视频演示

携程景点数据抓取及分析系统的设计与实现_哔哩哔哩_bilibili

系统介绍

摘 要：随着旅游业的快速发展，携程旅游平台已成为国内领先的在线旅游服务平台。然而，由于海量景点信息的不断涌现，如何高效地获取和分析这些数据以满足用户需求便成为了一大挑战。当前的爬取和分析系统普遍存在问题，如数据更新不及时、数据质量参差不齐、分析效果不尽如人意等。针对这些问题，本研究对携程景点数据爬取及分析系统进行了优化。首先，采用了高效的爬虫技术，使用合适的爬虫框架，提高了数据爬取速度和稳定性。其次，利用Python语言技术对爬取的数据进行清洗、去重或者情感分析，以提高数据质量和价值。最后，通过可视化工具将分析结果展示给用户，最后存入MySQL数据库当中，以便于用户做出更好的决策。总之，本研究对携程景点数据爬取及分析系统进行了全面优化，提高了系统性能和用户体验。

关键词：景点数据爬取；Python；MySQL

3  系统设计

（4） 在流程梳理清楚过后，进入到页面设计阶段，针对需求人员设计符合对应的页面效果，主要用简单、流畅、易懂的页面效果来设计，有便于用户的早期使用不会很费劲，后台人员的设计主要是实用性为主，不要过于花里胡哨的页面，这样让用户在维护过程中比较困难。

3.1系统总体设计

考虑到后期维护问题，从早期设计层面就需要有一定的标准性来规划。主要规则有：

简单性：操作上要采用最简单的方式，减少培训成本，操作人员见字便懂意。

针对性：定向开发针对系统的核心特点，专攻性强，避免混乱的程序让人难以懂得。

实用性：实用性是一个项目开发的主要目的，无论什么项目加入互联网就是为了方便实用。

一致性：页面美学设计风格一致，采用的元素和颜色以及风格必须保持一致。功能特征和代码命名规范。方便后期维护。

先进性：采用最新前卫主流技术和框架，适应编程技术的科技发展，让代码能更好的迭代和更新。

3.2系统功能模块

3.3 数据库逻辑结构设计

数据库的设计将会直接影响到项目的流程和便携性，如果没有设计好，甚至可能后期将和原设计功能背道而驰。最开始先梳理清楚项目流程和需求，在更具需求设计功能，建立二者的基础上设计数据库，既满足功能并且保留一定的拓展性，为后期修改保留一定空间，必然后期修改不便。

技术栈

1. 后端：Spring+SpringMVC+Mybatis
2. 前端：JSP+CSS+JavaScript+jQuery

使用说明

1. 使用Navicat或者其它工具，在mysql中创建对应名称的数据库，并导入项目的sql文件；
2. 使用IDEA/Eclipse/MyEclipse导入项目，Eclipse/MyEclipse导入时，若为maven项目请选择maven;
若为maven项目，导入成功后请执行maven clean;maven install命令，然后运行；
3. 将项目中springmvc-servlet.xml配置文件中的数据库配置改为自己的配置;
4. 运行项目，在浏览器中输入http://localhost:8080/ 登录

运行截图

 用户管理控制层:

package com.houserss.controller;import javax.servlet.http.HttpSession;import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;import com.houserss.common.Const;
import com.houserss.common.Const.Role;
import com.houserss.common.ServerResponse;
import com.houserss.pojo.User;
import com.houserss.service.IUserService;
import com.houserss.service.impl.UserServiceImpl;
import com.houserss.util.MD5Util;
import com.houserss.util.TimeUtils;
import com.houserss.vo.DeleteHouseVo;
import com.houserss.vo.PageInfoVo;/*** Created by admin*/
@Controller
@RequestMapping("/user/")
public class UserController {@Autowiredprivate IUserService iUserService;/*** 用户登录* @param username* @param password* @param session* @return*/@RequestMapping(value = "login.do",method = RequestMethod.POST)@ResponseBodypublic ServerResponse login(User user,String uvcode, HttpSession session){String code = (String)session.getAttribute("validationCode");if(StringUtils.isNotBlank(code)) {if(!code.equalsIgnoreCase(uvcode)) {return ServerResponse.createByErrorMessage("验证码不正确");}}ServerResponse response = iUserService.login(user.getUsername(),user.getPassword());if(response.isSuccess()){session.setAttribute(Const.CURRENT_USER,response.getData());}return response;}}

管理员管理控制层:

package com.sxl.controller.admin;import java.util.List;
import java.util.Map;import javax.servlet.http.HttpServletRequest;import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;import com.sxl.controller.MyController;@Controller("adminController")
@RequestMapping(value = "/admin")
public class AdminController extends MyController {@RequestMapping(value = "/index")public String frame(Model model, HttpServletRequest request)throws Exception {return "/admin/index";}@RequestMapping(value = "/main")public String main(Model model, HttpServletRequest request)throws Exception {return "/admin/main";}@RequestMapping(value = "/tj1")public String tj1(Model model, HttpServletRequest request)throws Exception {String sql="select DATE_FORMAT(insertDate,'%Y-%m-%d') dates,sum(allPrice) price from t_order order by DATE_FORMAT(insertDate,'%Y-%m-%d')  desc";List

修改密码业务逻辑:

package com.sxl.controller.admin;import java.util.Map;import javax.servlet.http.HttpServletRequest;import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;import com.sxl.controller.MyController;@Controller("userController")
@RequestMapping(value = "/user")
public class UserController extends MyController {@RequestMapping(value = "/index")public String frame(Model model, HttpServletRequest request)throws Exception {return "/user/index";}@RequestMapping(value = "/main")public String main(Model model, HttpServletRequest request)throws Exception {return "/user/main";}@RequestMapping(value = "/password")public String password(Model model, HttpServletRequest request)throws Exception {return "/user/password";}@RequestMapping(value = "/changePassword")public ResponseEntity loginSave(Model model,HttpServletRequest request,String oldPassword,String newPassword) throws Exception {Map user = getUser(request);if(oldPassword.equals(user.get("password").toString())){String sql="update t_user set password=? where id=?";db.update(sql, new Object[]{newPassword,user.get("id")});return renderData(true,"1",null);}else{return renderData(false,"1",null);}}@RequestMapping(value = "/mine")public String mine(Model model, HttpServletRequest request)throws Exception {
Map user =getUser(request);Map map = db.queryForMap("select * from t_user where id=?",new Object[]{user.get("id")});model.addAttribute("map", map);		return "/user/mine";}@RequestMapping(value = "/mineSave")public ResponseEntity mineSave(Model model,HttpServletRequest request,Long id,String username,String password,String name,String gh,String mobile) throws Exception{int result = 0;String sql="update t_user set name=?,gh=?,mobile=? where id=?";result = db.update(sql, new Object[]{name,gh,mobile,id});if(result==1){return renderData(true,"操作成功",null);}else{return renderData(false,"操作失败",null);}}}

通用管理模块:

package com.sxl.controller;import java.nio.charset.Charset;
import java.util.Locale;
import java.util.ResourceBundle;import javax.servlet.http.HttpServletRequest;import org.apache.commons.lang.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;import com.sxl.util.JacksonJsonUtil;
import com.sxl.util.StringUtil;
import com.sxl.util.SystemProperties;public class BaseController {public static final Long EXPIRES_IN = 1000 * 3600 * 24 * 1L;// 1天@Autowiredprivate SystemProperties systemProperties;/*** 获得配置文件内容*/public String getConfig(String key) {return systemProperties.getProperties(key);}/*** 返回服务器地址 like http://192.168.1.1:8441/UUBean/*/public String getHostUrl(HttpServletRequest request) {String hostName = request.getServerName();Integer hostPort = request.getServerPort();String path = request.getContextPath();if (hostPort == 80) {return "http://" + hostName + path + "/";} else {return "http://" + hostName + ":" + hostPort + path + "/";}}/**** 获取当前的website路径 String*/public static String getWebSite(HttpServletRequest request) {String returnUrl = request.getScheme() + "://"+ request.getServerName();if (request.getServerPort() != 80) {returnUrl += ":" + request.getServerPort();}returnUrl += request.getContextPath();return returnUrl;}/*** 初始化HTTP头.* * @return HttpHeaders*/public HttpHeaders initHttpHeaders() {HttpHeaders headers = new HttpHeaders();MediaType mediaType = new MediaType("text", "html",Charset.forName("utf-8"));headers.setContentType(mediaType);return headers;}/*** 返回 信息数据* * @param status* @param msg* @return*/public ResponseEntity renderMsg(Boolean status, String msg) {if (StringUtils.isEmpty(msg)) {msg = "";}String str = "{\"status\":\"" + status + "\",\"msg\":\"" + msg + "\"}";ResponseEntity responseEntity = new ResponseEntity(str,initHttpHeaders(), HttpStatus.OK);return responseEntity;}/*** 返回obj数据* * @param status* @param msg* @param obj* @return*/public ResponseEntity renderData(Boolean status, String msg,Object obj) {if (StringUtils.isEmpty(msg)) {msg = "";}StringBuffer sb = new StringBuffer();sb.append("{");sb.append("\"status\":\"" + status + "\",\"msg\":\"" + msg + "\",");sb.append("\"data\":" + JacksonJsonUtil.toJson(obj) + "");sb.append("}");ResponseEntity responseEntity = new ResponseEntity(sb.toString(), initHttpHeaders(), HttpStatus.OK);return responseEntity;}/**** 获取IP（如果是多级代理，则得到的是一串IP值）*/public static String getIpAddr(HttpServletRequest request) {String ip = request.getHeader("x-forwarded-for");if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {ip = request.getHeader("Proxy-Client-IP");}if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {ip = request.getHeader("WL-Proxy-Client-IP");}if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {ip = request.getRemoteAddr();}if (ip != null && ip.length() > 0) {String[] ips = ip.split(",");for (int i = 0; i < ips.length; i++) {if (!"unknown".equalsIgnoreCase(ips[i])) {ip = ips[i];break;}}}return ip;}/*** 国际化获得语言内容* * @param key*            语言key* @param args* @param argsSplit* @param defaultMessage* @param locale* @return*/public static String getLanguage(String key, String args, String argsSplit,String defaultMessage, String locale) {String language = "zh";String contry = "cn";String returnValue = defaultMessage;if (!StringUtil.isEmpty(locale)) {try {String[] localeArray = locale.split("_");language = localeArray[0];contry = localeArray[1];} catch (Exception e) {}}try {ResourceBundle resource = ResourceBundle.getBundle("lang.resource",new Locale(language, contry));returnValue = resource.getString(key);if (!StringUtil.isEmpty(args)) {String[] argsArray = args.split(argsSplit);for (int i = 0; i < argsArray.length; i++) {returnValue = returnValue.replace("{" + i + "}",argsArray[i]);}}} catch (Exception e) {}return returnValue;}
}

本文来自互联网用户投稿，文章观点仅代表作者本人，不代表本站立场，不承担相关法律责任。如若转载，请注明出处。 如若内容造成侵权/违法违规/事实不符，请点击【内容举报】进行投诉反馈！